-- Logs begin at Tue 2022-10-11 23:31:15 UTC, end at Wed 2022-10-12 00:44:22 UTC. -- Oct 12 00:00:41.378536 dv-cinder-849143-13-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Oct 12 00:00:41.385677 dv-cinder-849143-13-iscsi devstack@keystone.service[81734]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Oct 12 00:00:41.391395 dv-cinder-849143-13-iscsi devstack@keystone.service[81734]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Oct 12 00:00:41 2022] *** Oct 12 00:00:41.391395 dv-cinder-849143-13-iscsi devstack@keystone.service[81734]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 12 00:00:41.391395 dv-cinder-849143-13-iscsi devstack@keystone.service[81734]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 12 00:00:41.391395 dv-cinder-849143-13-iscsi devstack@keystone.service[81734]: nodename: dv-cinder-849143-13-iscsi Oct 12 00:00:41.391395 dv-cinder-849143-13-iscsi devstack@keystone.service[81734]: machine: x86_64 Oct 12 00:00:41.391395 dv-cinder-849143-13-iscsi devstack@keystone.service[81734]: clock source: unix Oct 12 00:00:41.391395 dv-cinder-849143-13-iscsi devstack@keystone.service[81734]: pcre jit disabled Oct 12 00:00:41.391395 dv-cinder-849143-13-iscsi devstack@keystone.service[81734]: detected number of CPU cores: 4 Oct 12 00:00:41.391395 dv-cinder-849143-13-iscsi devstack@keystone.service[81734]: current working directory: / Oct 12 00:00:41.391395 dv-cinder-849143-13-iscsi devstack@keystone.service[81734]: detected binary path: /usr/bin/uwsgi-core Oct 12 00:00:41.391395 dv-cinder-849143-13-iscsi devstack@keystone.service[81734]: your processes number limit is 47680 Oct 12 00:00:41.391395 dv-cinder-849143-13-iscsi devstack@keystone.service[81734]: your memory page size is 4096 bytes Oct 12 00:00:41.391395 dv-cinder-849143-13-iscsi devstack@keystone.service[81734]: detected max file descriptor number: 2048 Oct 12 00:00:41.391395 dv-cinder-849143-13-iscsi devstack@keystone.service[81734]: lock engine: pthread robust mutexes Oct 12 00:00:41.391395 dv-cinder-849143-13-iscsi devstack@keystone.service[81734]: thunder lock: enabled Oct 12 00:00:41.391395 dv-cinder-849143-13-iscsi devstack@keystone.service[81734]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Oct 12 00:00:41.391395 dv-cinder-849143-13-iscsi devstack@keystone.service[81734]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 12 00:00:41.428724 dv-cinder-849143-13-iscsi devstack@keystone.service[81734]: Python main interpreter initialized at 0x56355fb609c0 Oct 12 00:00:41.428724 dv-cinder-849143-13-iscsi devstack@keystone.service[81734]: python threads support enabled Oct 12 00:00:41.428724 dv-cinder-849143-13-iscsi devstack@keystone.service[81734]: your server socket listen backlog is limited to 100 connections Oct 12 00:00:41.428724 dv-cinder-849143-13-iscsi devstack@keystone.service[81734]: your mercy for graceful operations on workers is 90 seconds Oct 12 00:00:41.429096 dv-cinder-849143-13-iscsi devstack@keystone.service[81734]: mapped 403077 bytes (393 KB) for 2 cores Oct 12 00:00:41.429172 dv-cinder-849143-13-iscsi devstack@keystone.service[81734]: *** Operational MODE: preforking *** Oct 12 00:00:41.429418 dv-cinder-849143-13-iscsi devstack@keystone.service[81734]: *** uWSGI is running in multiple interpreter mode *** Oct 12 00:00:41.429418 dv-cinder-849143-13-iscsi devstack@keystone.service[81734]: spawned uWSGI master process (pid: 81734) Oct 12 00:00:41.429623 dv-cinder-849143-13-iscsi systemd[1]: Started Devstack devstack@keystone.service. Oct 12 00:00:41.430691 dv-cinder-849143-13-iscsi devstack@keystone.service[81734]: spawned uWSGI worker 1 (pid: 81746, cores: 1) Oct 12 00:00:41.430691 dv-cinder-849143-13-iscsi devstack@keystone.service[81734]: spawned uWSGI worker 2 (pid: 81747, cores: 1) Oct 12 00:00:41.430691 dv-cinder-849143-13-iscsi devstack@keystone.service[81734]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 12 00:00:45.714689 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 12 00:00:45.714689 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: from cryptography.utils import int_from_bytes Oct 12 00:00:45.715770 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 12 00:00:45.715770 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: from cryptography.utils import int_from_bytes Oct 12 00:00:45.716230 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 12 00:00:45.716230 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: from cryptography.utils import int_from_bytes Oct 12 00:00:45.716366 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 12 00:00:45.716366 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: from cryptography.utils import int_from_bytes Oct 12 00:00:45.766255 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 12 00:00:45.766435 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 12 00:00:45.766628 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] command line args: [] {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 12 00:00:45.766883 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 12 00:00:45.766976 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 12 00:00:45.767053 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ================================================================================ {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 12 00:00:45.767119 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 12 00:00:45.767250 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] command line args: [] {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 12 00:00:45.767349 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] admin_token = **** {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.767538 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 12 00:00:45.767620 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] config_dir = [] {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.767720 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ================================================================================ {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 12 00:00:45.767802 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.767916 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] config_source = [] {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.768002 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] admin_token = **** {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.768102 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] debug = True {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.768188 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] config_dir = [] {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.768303 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.768543 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.768543 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] config_source = [] {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.768659 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.768720 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] debug = True {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.768863 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.768966 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.769098 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.769209 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.769311 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.769419 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.769515 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.769629 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] list_limit = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.769733 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] log_config_append = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.769833 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.769928 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.770034 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] list_limit = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.770128 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] log_dir = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.770222 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] log_config_append = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.770327 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] log_file = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.770428 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.770530 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.770649 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] log_dir = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.770649 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] log_file = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.770833 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.770916 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.770992 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.771117 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.771198 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.771270 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.771344 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.771496 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.771578 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.771703 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.771777 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.771899 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.771981 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.772098 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.772177 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.772328 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.772409 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.772560 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.772679 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.772790 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.772873 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.773021 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.773097 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.773253 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.773347 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.773499 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.773644 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.773726 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.774535 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.774624 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.36/identity {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.774732 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] publish_errors = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.774967 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.775086 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.36/identity {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.775193 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] publish_errors = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.775270 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.775423 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.775509 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.775630 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.775708 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.775837 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.775931 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.776079 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.776166 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] standard_threads = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.776310 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.776386 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.776538 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] standard_threads = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.776614 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.776718 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.776800 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.776896 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] use_journal = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.776978 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.777073 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] use_json = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.777148 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.777248 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] use_stderr = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.777333 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] use_journal = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.777424 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] use_syslog = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.777500 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] use_json = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.777630 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] use_stderr = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.777711 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.777804 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] use_syslog = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.777924 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.778032 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 00:00:45.778109 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.778272 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.778355 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.778505 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.778663 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.778761 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.778836 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.778997 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.779072 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.779244 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.779327 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.779502 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.779579 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.779747 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.779823 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.779982 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.780066 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.780226 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.780303 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.780471 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.780600 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.780713 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.780790 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.780943 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.781019 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.781186 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.781274 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.781420 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.781503 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.781655 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.781737 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.781898 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.781979 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.782133 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.782211 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.782382 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.782462 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.782618 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.782694 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.782859 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.782941 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.783093 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.783175 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.783343 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.783454 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.783623 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.783700 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.783857 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.783933 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.784089 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.784171 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.784322 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.784399 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.784532 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.784615 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.784765 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.784847 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.784997 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.785075 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.785240 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.785315 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.785472 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.785555 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.785697 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.785774 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.785925 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.786001 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.786168 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.786245 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.786411 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.786489 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.786641 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.786743 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.786878 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.786990 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.787094 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.787189 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.787448 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.787577 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.787663 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.787793 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.787875 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.787951 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.788067 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.788140 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.788293 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.788384 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.788458 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.788574 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.788649 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.788774 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.788888 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.788978 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.789119 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.789197 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.789353 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.790083 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.790232 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.790329 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.790413 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.790521 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.790611 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.790732 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.790813 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.790884 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.791008 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.791080 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.791235 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.791313 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.791491 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.791580 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.791727 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.791808 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.791957 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.792037 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.792189 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.792269 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.792417 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.792495 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.792655 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.792731 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.792885 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.792961 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.793121 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.793208 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.793353 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.793431 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.793590 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.793678 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.793855 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.793953 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.794047 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.794124 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.794280 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.794358 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.794514 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.794603 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.794787 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.794869 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.795024 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.795099 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.795260 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.795344 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.795522 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.795675 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.795754 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.795824 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.795968 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.796117 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.796191 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.796261 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.796432 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.796509 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.796665 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.796751 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.796901 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.796985 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.797141 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.797218 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.797375 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.797464 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.797611 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.797695 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.797842 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.797925 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.798076 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.798183 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.798304 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.798384 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.798536 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.798614 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.798770 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.798848 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.799025 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.799103 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.799261 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.799337 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.799522 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.799600 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.799760 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.799838 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.799995 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.800072 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.800231 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.800307 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.800464 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.800541 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.800695 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.800770 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.800926 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.801003 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.801155 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.801232 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.801381 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.801458 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.801608 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.801685 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.801839 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.801916 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.802068 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.802145 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.802306 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.802383 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.802539 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.802618 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.802784 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.802861 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.803019 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.803098 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.803269 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.803388 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.803530 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] auth.external = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.803647 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.803765 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.803845 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.803990 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] auth.external = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.804076 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.804220 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.804300 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.804500 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] auth.password = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.804598 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.804802 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.804926 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] auth.token = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.805012 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] auth.password = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.805091 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.805194 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] auth.token = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.805273 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.805431 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.805509 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.805665 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.805744 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.805908 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.805986 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.806136 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.806224 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.806373 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.806451 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.806610 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] credential.caching = True {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.806688 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.806836 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.806921 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.807069 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] credential.caching = True {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.807159 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.807314 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.807472 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.807584 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.807674 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.807826 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.807903 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.808062 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.808139 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.808300 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.808387 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.808534 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.808613 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.808776 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.808856 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.809025 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.809110 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.809260 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.809337 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.809510 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.809589 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.809786 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.809940 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.810015 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.810084 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.810271 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.810349 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.810505 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] federation.caching = True {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.810590 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.810743 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.810821 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.810974 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] federation.caching = True {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.811051 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.811211 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.811297 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.811467 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.811547 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.811712 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.811789 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.811942 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.814067 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.814179 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.814258 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.814330 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.814400 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.814470 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.814548 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.814620 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.814690 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.814760 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.814830 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.814906 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.814985 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] identity.caching = True {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.815058 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.815133 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.815205 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] identity.caching = True {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.815274 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.815344 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.815497 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.815581 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.815654 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.815724 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.815782 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.815899 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.815984 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.816057 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.816128 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.816196 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.816266 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.816335 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.816412 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.816484 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.816554 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.816623 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.816693 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.816770 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.816842 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.816912 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.816982 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.817052 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.817121 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.817198 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.817270 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.817345 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.817422 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.817493 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.817571 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.817650 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.817722 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.817792 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.817899 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.817973 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.818044 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.818174 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.818258 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.818331 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.818424 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.818559 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.818658 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.818797 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.818878 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.819028 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.819106 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.819262 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.819341 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.819525 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.819606 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.819763 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.819842 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.820012 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.820090 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.820243 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.820323 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.820489 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.820567 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.820760 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.820839 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.820997 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.821077 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.821226 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.821303 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.821465 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.821543 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.821691 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.821776 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.821927 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.822004 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.822152 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.822229 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.822387 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.822467 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.822621 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.822699 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.822858 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.822967 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.823092 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.823223 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.823465 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.823548 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.823718 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.823796 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.823951 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.824033 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.824185 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.824262 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.824417 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.824495 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.824665 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.824744 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.824903 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.824979 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.825142 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.825219 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.825373 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.825452 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.user = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.825609 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.825687 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.825839 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.user = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.825918 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.826077 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.826156 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.826323 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.826400 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.826552 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.826629 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.826798 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.826878 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.827033 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.827110 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.827267 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.827344 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.827520 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.827600 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.827750 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.827828 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.827984 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.828071 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.828214 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.828292 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.828445 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.828523 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.828673 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.828760 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.828917 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.828995 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.829161 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.829239 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.829405 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.829490 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.829639 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.829717 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.829879 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.829956 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.830111 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.830208 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.830345 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.830423 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.830588 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.830705 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.830809 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.830899 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.831042 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.831119 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.831275 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.831353 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.831539 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.831630 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.831782 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.831860 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.832014 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.832091 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.832258 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.832406 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.832482 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.832557 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.832713 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.832792 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.832947 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.833035 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.833183 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.833261 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.833417 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.833495 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.833648 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.833733 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.833886 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.833963 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.834113 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.834190 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] resource.caching = True {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.834348 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.834434 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.834586 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] resource.caching = True {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.834665 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.834833 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.834909 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.835084 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.835209 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.835318 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.835563 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.835563 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.835687 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.835816 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.835934 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.836046 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.836125 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.836799 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.836884 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.837042 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.837128 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] role.caching = True {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.837282 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.837359 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] role.driver = sql {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.837510 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.837586 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] role.caching = True {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.837754 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] role.driver = sql {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.837839 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.837985 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.838062 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.838232 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.838309 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.838477 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.838562 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.838720 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.838797 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.838964 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.839040 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.839206 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.839291 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.839473 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.839555 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.839703 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.839780 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.839951 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.840041 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.840178 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.840254 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.840421 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.840498 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.840664 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.840742 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.840906 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.841056 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.841138 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.841292 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.841521 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.841600 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.841782 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.841867 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.842040 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.842145 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.842232 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.842306 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.842469 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.842549 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.842721 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.842799 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.842956 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.843038 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.843197 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.843274 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.843454 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.843535 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.847253 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.847334 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.847413 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.847478 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.847478 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.847597 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.847669 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.847741 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.847810 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.847879 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.847947 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.848016 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.848093 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.848165 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.848165 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.848300 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.848370 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] token.caching = True {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.848444 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.848444 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.848585 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.848656 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] token.caching = True {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.848725 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.848793 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.848869 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.848940 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.849009 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.849079 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.849147 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.849216 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.849293 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.849363 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.849432 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.849500 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.849569 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.849637 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.849715 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.849786 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.849855 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.849924 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.849992 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.850061 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.850137 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.850207 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.850276 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.850344 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.850413 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.850487 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.850564 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.850643 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.850725 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.850796 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.850872 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.850950 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.851022 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] database.connection = **** {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.851108 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.851168 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.851233 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] database.connection = **** {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.851292 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.851349 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.851445 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.851505 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.851562 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.851630 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.851695 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.851753 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.851809 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.851865 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.851988 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.852688 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.852770 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.852829 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.852887 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.852887 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.853000 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.853077 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.853154 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.853230 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.853301 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.853369 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.853445 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.853522 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.853597 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.853691 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.853770 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.853922 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.853998 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.854162 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.854240 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.854390 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.854467 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.854560 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 12 00:00:45.854636 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.854856 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 00:00:45.855025 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 12 00:00:45.883693 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81746) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 00:00:45.883833 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81747) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 00:00:45.883913 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81746) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 00:00:45.883991 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81747) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 00:00:45.884133 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81746) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 00:00:45.884217 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81747) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 00:00:45.892642 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81746) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 00:00:45.892739 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81747) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 00:00:45.921206 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81746) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 00:00:45.921521 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81746) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 00:00:45.922428 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81747) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 00:00:45.922663 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81747) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 00:00:45.979759 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81746) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 00:00:45.980468 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81747) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 00:00:46.028566 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81746) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 00:00:46.028800 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81746) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 00:00:46.030394 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81747) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 00:00:46.030700 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81747) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 00:00:46.188396 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81746) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 00:00:46.188623 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81746) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 00:00:46.188825 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81746) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 00:00:46.189045 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81746) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 00:00:46.189260 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81746) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 00:00:46.189472 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81746) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 00:00:46.190781 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81747) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 00:00:46.191083 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81747) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 00:00:46.191390 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81747) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 00:00:46.191685 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81747) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 00:00:46.191971 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81747) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 00:00:46.192252 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81747) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 00:00:46.217340 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81747) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 00:00:46.217666 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81747) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 00:00:46.217959 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81747) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 00:00:46.218282 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81746) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 00:00:46.218376 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81747) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 00:00:46.218514 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81746) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 00:00:46.218660 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81747) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 00:00:46.218745 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81746) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 00:00:46.218935 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81746) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 00:00:46.219018 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81747) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 00:00:46.219147 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81746) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 00:00:46.219318 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81747) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 00:00:46.219420 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81746) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 00:00:46.219578 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81746) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 00:00:46.231205 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81747) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 00:00:46.231876 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81746) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 00:00:46.244494 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81747) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 00:00:46.244730 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81747) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 00:00:46.244818 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81746) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 00:00:46.244960 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81747) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 00:00:46.245042 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81746) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 00:00:46.245184 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81747) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 00:00:46.245267 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81746) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 00:00:46.245401 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81747) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 00:00:46.245482 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81746) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 00:00:46.245613 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81747) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 00:00:46.245704 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81746) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 00:00:46.245825 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81746) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 00:00:46.245904 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81747) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 00:00:46.246039 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81746) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 00:00:46.262704 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.262914 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 12 00:00:46.263040 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.263234 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 12 00:00:46.263326 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.263484 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 12 00:00:46.263608 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.263731 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.263814 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 12 00:00:46.263995 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.264073 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.264255 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.264342 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.264511 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.264590 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.264802 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.274146 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.274380 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.274477 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.274659 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.274740 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.274931 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.275012 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.275213 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.281914 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81747) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 00:00:46.282141 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81746) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 00:00:46.284848 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81747) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 00:00:46.285047 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81746) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 00:00:46.285129 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.285331 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.285412 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.285616 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.285697 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.285900 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.285990 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.286185 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.286266 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.286446 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.286527 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.286724 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.286844 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.286990 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.287076 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.287258 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.287341 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.287557 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.287644 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.287824 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.311263 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.311393 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.312750 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81747) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 00:00:46.312836 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81746) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 00:00:46.313036 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.313119 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.317695 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81746) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 00:00:46.317804 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81747) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 00:00:46.317976 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.318061 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.318255 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.318338 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.325081 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81746) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 00:00:46.325243 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81747) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 00:00:46.325362 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.325528 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.330929 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81746) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 00:00:46.331151 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81747) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 00:00:46.331250 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.331476 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.331570 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.331774 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.331929 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.332087 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.332221 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.332354 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.332484 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.332615 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.332745 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.332882 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.333012 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.333141 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.348841 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.349118 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.349271 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.349404 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.352997 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81746) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 00:00:46.353326 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81747) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 00:00:46.356246 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.356541 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.356672 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.356851 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.360727 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81746) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 00:00:46.361073 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81747) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 00:00:46.363848 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81746) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 00:00:46.364272 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81747) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 00:00:46.367043 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.367343 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.368094 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.368227 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.368401 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.368529 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.368690 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.368816 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.368976 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.369115 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.369252 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.369527 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.385186 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81746) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 00:00:46.385467 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.385735 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.385952 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81747) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 00:00:46.386079 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.386243 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.386779 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.387044 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.393115 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.393939 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.394596 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.395514 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.396159 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.396431 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.397043 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.397321 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.399296 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81746) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 00:00:46.399606 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.399889 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.400083 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81747) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 00:00:46.400210 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.400371 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.400580 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.400709 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.400928 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.401328 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.411620 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81746) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 00:00:46.411910 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.412192 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.412470 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.412597 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81747) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 00:00:46.412825 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.412950 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.413112 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.413480 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.413480 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.413787 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.413787 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.414071 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.414366 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.428648 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81746) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 00:00:46.429892 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81747) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 00:00:46.432064 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81746) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 00:00:46.433276 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81747) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 00:00:46.434939 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.436215 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.436430 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.437796 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.437929 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81746) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 00:00:46.438152 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.438594 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.439329 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81747) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 00:00:46.439639 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.440043 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.445269 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.446901 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81746) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 00:00:46.447000 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.448458 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81747) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 00:00:46.449786 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.450181 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.450467 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.450746 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.451321 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.451740 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.452029 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.452313 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.458483 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81746) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 00:00:46.458771 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.459051 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.460148 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81747) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 00:00:46.460442 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.460723 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.465934 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81746) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 00:00:46.466216 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.472483 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.472483 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.472483 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.472483 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.472483 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.472910 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81747) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 00:00:46.472998 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.473073 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.473145 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.473217 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.473288 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.473367 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.473440 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.473511 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.473582 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.473652 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.473731 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.473804 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.473874 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.473945 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.473945 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.473945 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.473945 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.473945 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.473945 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.474318 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 00:00:46.503732 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x56355fb609c0 pid: 81746 (default app) Oct 12 00:00:46.505538 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x56355fb609c0 pid: 81747 (default app) Oct 12 00:00:46.507025 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c54904c-d3b5-41b2-b1b3-77b0b1e3464a None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:00:46.507310 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c54904c-d3b5-41b2-b1b3-77b0b1e3464a None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:00:46.507612 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c54904c-d3b5-41b2-b1b3-77b0b1e3464a None None] PATH_INFO: `/v3` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:00:46.512629 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1/1] 10.222.0.36 () {50 vars in 791 bytes} [Wed Oct 12 00:00:46 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:00:54.149708 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f24b026-9917-4ec0-b825-9c35b5143b2f None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:00:54.150220 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f24b026-9917-4ec0-b825-9c35b5143b2f None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:00:54.150220 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f24b026-9917-4ec0-b825-9c35b5143b2f None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:00:54.154517 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1/2] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:00:54 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:00:54.159636 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5b96c9-47d7-46f9-9230-c374aba35625 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:00:54.160078 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5b96c9-47d7-46f9-9230-c374aba35625 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:00:54.160173 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5b96c9-47d7-46f9-9230-c374aba35625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:00:54.231177 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: INFO dbcounter [None req-1b5b96c9-47d7-46f9-9230-c374aba35625 None None] Registered counter for database keystone Oct 12 00:00:54.250561 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG oslo_db.sqlalchemy.engines [None req-1b5b96c9-47d7-46f9-9230-c374aba35625 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81746) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 12 00:00:54.256888 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writer thread running {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 12 00:00:54.437455 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG passlib.handlers.bcrypt [None req-1b5b96c9-47d7-46f9-9230-c374aba35625 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81746) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 12 00:00:54.437764 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG passlib.handlers.bcrypt [None req-1b5b96c9-47d7-46f9-9230-c374aba35625 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81746) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 12 00:00:54.495670 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-1b5b96c9-47d7-46f9-9230-c374aba35625 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:00:54.497288 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-1b5b96c9-47d7-46f9-9230-c374aba35625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:00:54.624373 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 2/3] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:00:54 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 466 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 12 00:00:55.067158 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f24b026-9917-4ec0-b825-9c35b5143b2f None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:00:55.135512 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: INFO dbcounter [None req-5f24b026-9917-4ec0-b825-9c35b5143b2f None None] Registered counter for database keystone Oct 12 00:00:55.156512 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG oslo_db.sqlalchemy.engines [None req-5f24b026-9917-4ec0-b825-9c35b5143b2f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81747) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 12 00:00:55.162599 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writer thread running {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 12 00:00:55.280333 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f24b026-9917-4ec0-b825-9c35b5143b2f None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:00:55.284512 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e32c4e8-a2ed-4b2c-b1c9-e7e8424617b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:00:55.285134 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e32c4e8-a2ed-4b2c-b1c9-e7e8424617b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:00:55.285410 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e32c4e8-a2ed-4b2c-b1c9-e7e8424617b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:00:55.285677 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e32c4e8-a2ed-4b2c-b1c9-e7e8424617b0 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:00:55.294233 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e32c4e8-a2ed-4b2c-b1c9-e7e8424617b0 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:00:55.346582 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: WARNING keystone.common.rbac_enforcer.enforcer [None req-2e32c4e8-a2ed-4b2c-b1c9-e7e8424617b0 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 12 00:00:55.346930 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e32c4e8-a2ed-4b2c-b1c9-e7e8424617b0 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:00:55.356185 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 2/4] 10.222.0.36 () {58 vars in 1324 bytes} [Wed Oct 12 00:00:55 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 12 00:00:55.362033 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5b96c9-47d7-46f9-9230-c374aba35625 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:00:55.379155 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5b96c9-47d7-46f9-9230-c374aba35625 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:00:55.383646 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacbad2a-6f74-4c77-85d9-29fc1a7627c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:00:55.385329 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacbad2a-6f74-4c77-85d9-29fc1a7627c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:00:55.385609 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacbad2a-6f74-4c77-85d9-29fc1a7627c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:00:55.385883 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacbad2a-6f74-4c77-85d9-29fc1a7627c1 admin admin] PATH_INFO: `/v3/projects/392dba9b727b479aa54a2746402ded73` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:00:55.387770 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacbad2a-6f74-4c77-85d9-29fc1a7627c1 admin admin] RBAC: Authorizing `identity:get_project(project_id=392dba9b727b479aa54a2746402ded73)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:00:55.440020 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: WARNING keystone.common.rbac_enforcer.enforcer [None req-dacbad2a-6f74-4c77-85d9-29fc1a7627c1 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 12 00:00:55.440363 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacbad2a-6f74-4c77-85d9-29fc1a7627c1 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:00:55.444834 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 3/5] 10.222.0.36 () {56 vars in 1207 bytes} [Wed Oct 12 00:00:55 2022] GET /identity/v3/projects/392dba9b727b479aa54a2746402ded73 => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:00:57.529343 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c860b886-1400-465d-80d7-bb42039516ec None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:00:57.529868 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c860b886-1400-465d-80d7-bb42039516ec None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:00:57.529868 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c860b886-1400-465d-80d7-bb42039516ec None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:00:57.533939 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 3/6] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:00:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:00:57.539292 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca916e7-ffb1-4f15-8b85-fac0d64144ed None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:00:57.539745 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca916e7-ffb1-4f15-8b85-fac0d64144ed None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:00:57.539877 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca916e7-ffb1-4f15-8b85-fac0d64144ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:00:57.559692 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-5ca916e7-ffb1-4f15-8b85-fac0d64144ed None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:00:57.561223 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-5ca916e7-ffb1-4f15-8b85-fac0d64144ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:00:57.569021 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 4/7] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:00:57 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 12 00:00:57.831946 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c860b886-1400-465d-80d7-bb42039516ec None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:00:57.843839 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c860b886-1400-465d-80d7-bb42039516ec None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:00:57.847909 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f92b2c1-88b3-403c-ae73-02a59c79e44d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:00:57.848500 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f92b2c1-88b3-403c-ae73-02a59c79e44d admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:00:57.848773 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f92b2c1-88b3-403c-ae73-02a59c79e44d admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:00:57.849037 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f92b2c1-88b3-403c-ae73-02a59c79e44d admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:00:57.856500 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f92b2c1-88b3-403c-ae73-02a59c79e44d admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:00:57.858250 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f92b2c1-88b3-403c-ae73-02a59c79e44d admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:00:57.866818 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 4/8] 10.222.0.36 () {58 vars in 1324 bytes} [Wed Oct 12 00:00:57 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 12 00:00:57.872346 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca916e7-ffb1-4f15-8b85-fac0d64144ed None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:00:57.881288 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca916e7-ffb1-4f15-8b85-fac0d64144ed None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:00:57.886461 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cdae20-2ccd-4858-8b7f-773268f4f5f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:00:57.887081 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cdae20-2ccd-4858-8b7f-773268f4f5f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:00:57.887381 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cdae20-2ccd-4858-8b7f-773268f4f5f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:00:57.887659 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cdae20-2ccd-4858-8b7f-773268f4f5f9 admin admin] PATH_INFO: `/v3/users/1fcb9022bfe743ae9405fc8470a5e016` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:00:57.888950 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cdae20-2ccd-4858-8b7f-773268f4f5f9 admin admin] RBAC: Authorizing `identity:get_user(user_id=1fcb9022bfe743ae9405fc8470a5e016)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:00:57.890765 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cdae20-2ccd-4858-8b7f-773268f4f5f9 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:00:57.892338 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 5/9] 10.222.0.36 () {56 vars in 1198 bytes} [Wed Oct 12 00:00:57 2022] GET /identity/v3/users/1fcb9022bfe743ae9405fc8470a5e016 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:00.068925 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea5c0a3-dcc6-446d-adda-d3b04edf1a94 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:00.069447 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea5c0a3-dcc6-446d-adda-d3b04edf1a94 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:00.069447 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea5c0a3-dcc6-446d-adda-d3b04edf1a94 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:00.070790 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 5/10] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:01:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:01:00.076542 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532ff804-3a7b-41b9-bf3f-ed35345828c1 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:00.076970 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532ff804-3a7b-41b9-bf3f-ed35345828c1 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:00.077095 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532ff804-3a7b-41b9-bf3f-ed35345828c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:00.094877 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-532ff804-3a7b-41b9-bf3f-ed35345828c1 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:01:00.096098 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-532ff804-3a7b-41b9-bf3f-ed35345828c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:01:00.114417 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cc8250-3712-4751-9e0a-488b9e2ec0cd None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:00.114698 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cc8250-3712-4751-9e0a-488b9e2ec0cd None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:00.114972 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cc8250-3712-4751-9e0a-488b9e2ec0cd None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:00.116321 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 6/11] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:01:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 00:01:00.121791 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eef6d1-74fb-4288-94c2-83e134a419b4 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:00.122096 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eef6d1-74fb-4288-94c2-83e134a419b4 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:00.122368 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eef6d1-74fb-4288-94c2-83e134a419b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:00.164076 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG passlib.handlers.bcrypt [None req-40eef6d1-74fb-4288-94c2-83e134a419b4 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81747) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 12 00:01:00.164456 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG passlib.handlers.bcrypt [None req-40eef6d1-74fb-4288-94c2-83e134a419b4 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81747) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 12 00:01:00.177243 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 6/12] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:01:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 00:01:00.213686 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-40eef6d1-74fb-4288-94c2-83e134a419b4 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:01:00.215275 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-40eef6d1-74fb-4288-94c2-83e134a419b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:01:00.316365 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 7/13] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:01:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 12 00:01:00.439589 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532ff804-3a7b-41b9-bf3f-ed35345828c1 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:00.464025 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:00.544100 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532ff804-3a7b-41b9-bf3f-ed35345828c1 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:00.577003 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f097bc-7262-438c-a243-85af972668eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:00.577631 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f097bc-7262-438c-a243-85af972668eb None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:00.577905 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f097bc-7262-438c-a243-85af972668eb None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:00.578174 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f097bc-7262-438c-a243-85af972668eb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:00.585441 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eef6d1-74fb-4288-94c2-83e134a419b4 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:00.589676 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f097bc-7262-438c-a243-85af972668eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 00:01:00.598663 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-92f097bc-7262-438c-a243-85af972668eb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 00:01:00.598663 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:01:00.598663 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:01:00.598663 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:01:00.598663 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:01:00.598663 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:01:00.598663 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:01:00.598663 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:01:00.598663 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:01:00.598663 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:01:00.598663 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:01:00.598663 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:01:00.598663 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 00:01:00.598663 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 00:01:00.598663 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 00:01:00.598663 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 00:01:00.599533 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 00:01:00.599533 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 00:01:00.599533 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 00:01:00.599533 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 00:01:00.599533 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:01:00.599533 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:01:00.599533 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:01:00.599533 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:01:00.599533 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:01:00.599533 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:01:00.599533 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:01:00.599533 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:01:00.599533 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:01:00.599533 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:01:00.599533 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:01:00.599533 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:01:00.600306 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:01:00.600306 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:01:00.600306 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:01:00.600306 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:01:00.600306 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 00:01:00.600306 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 00:01:00.600306 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 00:01:00.600306 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 00:01:00.600306 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 00:01:00.600306 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 00:01:00.600306 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 00:01:00.600306 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:01:00.600306 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 7/14] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Oct 12 00:01:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 00:01:00.606115 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f097bc-7262-438c-a243-85af972668eb None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:00.685813 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eef6d1-74fb-4288-94c2-83e134a419b4 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:00.686315 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:00.688079 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:00.706726 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f097bc-7262-438c-a243-85af972668eb None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:00.721649 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0188b03-f556-42ca-85cb-6b80af6ce48f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:00.722513 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0188b03-f556-42ca-85cb-6b80af6ce48f None admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:00.722789 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0188b03-f556-42ca-85cb-6b80af6ce48f None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:00.723054 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0188b03-f556-42ca-85cb-6b80af6ce48f None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:00.724045 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0188b03-f556-42ca-85cb-6b80af6ce48f None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:00.725486 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0188b03-f556-42ca-85cb-6b80af6ce48f None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:00.738476 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f2bbde-4759-4169-a697-db56418d0a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:00.739088 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f2bbde-4759-4169-a697-db56418d0a13 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:00.739384 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f2bbde-4759-4169-a697-db56418d0a13 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:00.739682 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f2bbde-4759-4169-a697-db56418d0a13 None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:00.740355 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f2bbde-4759-4169-a697-db56418d0a13 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:01:00.740657 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f2bbde-4759-4169-a697-db56418d0a13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:00.742132 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f2bbde-4759-4169-a697-db56418d0a13 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:00.754338 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 8/15] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Oct 12 00:01:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 00:01:00.760120 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f2bbde-4759-4169-a697-db56418d0a13 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:00.801530 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.sql.core [None req-b0188b03-f556-42ca-85cb-6b80af6ce48f None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 12 00:01:00.801530 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 12 00:01:00.801530 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [parameters: {'id': '16306dfc2d834d8ebe7e3b5528c614d0', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 12 00:01:00.801530 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81747) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 12 00:01:00.822209 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-b0188b03-f556-42ca-85cb-6b80af6ce48f None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 12 00:01:00.822209 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:01:00.822209 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 12 00:01:00.822209 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 12 00:01:00.822209 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 12 00:01:00.822209 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 12 00:01:00.822209 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 12 00:01:00.822209 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application result = self._query(query) Oct 12 00:01:00.822209 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 12 00:01:00.822209 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application conn.query(q) Oct 12 00:01:00.822209 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 12 00:01:00.822209 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 12 00:01:00.822209 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 12 00:01:00.822209 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application result.read() Oct 12 00:01:00.823070 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 12 00:01:00.823070 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 12 00:01:00.823070 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 12 00:01:00.823070 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 12 00:01:00.823070 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 12 00:01:00.823070 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 12 00:01:00.823070 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 12 00:01:00.823070 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 12 00:01:00.823070 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 12 00:01:00.823070 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:01:00.823070 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 12 00:01:00.823070 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:01:00.823070 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:01:00.823070 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 12 00:01:00.823070 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 12 00:01:00.823070 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 12 00:01:00.823070 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 12 00:01:00.823070 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 12 00:01:00.824021 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application next(self.gen) Oct 12 00:01:00.824021 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 12 00:01:00.824021 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application yield resource Oct 12 00:01:00.824021 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 12 00:01:00.824021 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application next(self.gen) Oct 12 00:01:00.824021 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 12 00:01:00.824021 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application self.session.rollback() Oct 12 00:01:00.824021 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 12 00:01:00.824021 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application self.force_reraise() Oct 12 00:01:00.824021 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 12 00:01:00.824021 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise self.value Oct 12 00:01:00.824021 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 12 00:01:00.824021 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 12 00:01:00.824021 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 12 00:01:00.824021 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application session.commit() Oct 12 00:01:00.824021 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 12 00:01:00.824021 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 12 00:01:00.824911 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 12 00:01:00.824911 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application self._prepare_impl() Oct 12 00:01:00.824911 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 12 00:01:00.824911 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application self.session.flush() Oct 12 00:01:00.824911 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 12 00:01:00.824911 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application self._flush(objects) Oct 12 00:01:00.824911 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 12 00:01:00.824911 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 12 00:01:00.824911 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 12 00:01:00.824911 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application compat.raise_( Oct 12 00:01:00.824911 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 12 00:01:00.824911 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception Oct 12 00:01:00.824911 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 12 00:01:00.824911 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application flush_context.execute() Oct 12 00:01:00.824911 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 12 00:01:00.824911 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rec.execute(self) Oct 12 00:01:00.824911 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 12 00:01:00.825779 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 12 00:01:00.825779 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 12 00:01:00.825779 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 12 00:01:00.825779 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 12 00:01:00.825779 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 12 00:01:00.825779 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 12 00:01:00.825779 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 12 00:01:00.825779 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 12 00:01:00.825779 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 12 00:01:00.825779 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 12 00:01:00.825779 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 12 00:01:00.825779 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 12 00:01:00.825779 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 12 00:01:00.825779 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 12 00:01:00.825779 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 12 00:01:00.826564 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 12 00:01:00.826564 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception Oct 12 00:01:00.826564 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 12 00:01:00.826564 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 12 00:01:00.826564 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 12 00:01:00.826564 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 12 00:01:00.826564 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 12 00:01:00.826564 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application result = self._query(query) Oct 12 00:01:00.826564 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 12 00:01:00.826564 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application conn.query(q) Oct 12 00:01:00.826564 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 12 00:01:00.826564 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 12 00:01:00.826564 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 12 00:01:00.826564 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application result.read() Oct 12 00:01:00.826564 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 12 00:01:00.826564 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 12 00:01:00.826564 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 12 00:01:00.827329 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 12 00:01:00.827329 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 12 00:01:00.827329 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 12 00:01:00.827329 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 12 00:01:00.827329 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 12 00:01:00.827329 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 12 00:01:00.827329 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 12 00:01:00.827329 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [parameters: {'id': '16306dfc2d834d8ebe7e3b5528c614d0', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 12 00:01:00.827329 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 12 00:01:00.827329 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:01:00.827329 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 00:01:00.827329 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:01:00.827329 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:01:00.827329 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 12 00:01:00.827329 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 12 00:01:00.828056 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 12 00:01:00.828056 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 12 00:01:00.828056 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 12 00:01:00.828056 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:01:00.828056 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 00:01:00.828056 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:01:00.828056 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:01:00.828056 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:01:00.828056 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:01:00.828056 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:01:00.828056 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:01:00.828056 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:01:00.828056 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:01:00.828056 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:01:00.828056 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:01:00.828056 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:01:00.828056 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:01:00.828873 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 12 00:01:00.828873 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 12 00:01:00.828873 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:01:00.828873 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:01:00.828873 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 12 00:01:00.828873 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 12 00:01:00.828873 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:01:00.828873 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:01:00.828873 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 12 00:01:00.828873 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 12 00:01:00.828873 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 12 00:01:00.828873 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:01:00.828873 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 8/16] 10.222.0.36 () {60 vars in 1137 bytes} [Wed Oct 12 00:01:00 2022] POST /identity/v3/domains => generated 190 bytes in 238 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 12 00:01:00.829525 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0188b03-f556-42ca-85cb-6b80af6ce48f None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:00.862942 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f2bbde-4759-4169-a697-db56418d0a13 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:00.895354 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b22702-5570-429b-a41f-e0c0402a57c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:00.896108 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b22702-5570-429b-a41f-e0c0402a57c7 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:00.896388 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b22702-5570-429b-a41f-e0c0402a57c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:00.896661 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b22702-5570-429b-a41f-e0c0402a57c7 None admin] PATH_INFO: `/v3/users/1fcb9022bfe743ae9405fc8470a5e016` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:00.897810 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b22702-5570-429b-a41f-e0c0402a57c7 None admin] RBAC: Authorizing `identity:get_user(user_id=1fcb9022bfe743ae9405fc8470a5e016)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:00.899829 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b22702-5570-429b-a41f-e0c0402a57c7 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:00.901606 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 9/17] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Oct 12 00:01:00 2022] GET /identity/v3/users/1fcb9022bfe743ae9405fc8470a5e016 => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 00:01:00.906748 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b22702-5570-429b-a41f-e0c0402a57c7 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:00.921262 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:00.931227 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0188b03-f556-42ca-85cb-6b80af6ce48f None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:00.964499 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bc551e-2f3d-4a08-a65f-20b1a4fe0e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:00.965316 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc551e-2f3d-4a08-a65f-20b1a4fe0e8a None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:00.965597 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc551e-2f3d-4a08-a65f-20b1a4fe0e8a None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:00.965866 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc551e-2f3d-4a08-a65f-20b1a4fe0e8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:00.984102 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bc551e-2f3d-4a08-a65f-20b1a4fe0e8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:00.985567 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bc551e-2f3d-4a08-a65f-20b1a4fe0e8a None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:01.001898 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-d5bc551e-2f3d-4a08-a65f-20b1a4fe0e8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:01:01.001898 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:01:01.001898 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:01:01.001898 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:01:01.001898 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:01:01.001898 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:01:01.001898 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:01:01.001898 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:01:01.001898 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:01:01.001898 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:01:01.001898 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:01:01.001898 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:01:01.001898 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 00:01:01.001898 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 00:01:01.001898 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 00:01:01.001898 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 00:01:01.002859 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:01:01.002859 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:01:01.002859 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:01:01.002859 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:01:01.002859 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:01:01.002859 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:01:01.002859 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:01:01.002859 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:01:01.002859 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:01:01.002859 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:01:01.002859 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:01:01.002859 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:01:01.002859 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:01:01.002859 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:01:01.002859 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:01:01.002859 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:01:01.002859 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 00:01:01.003774 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 00:01:01.003774 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:01:01.003774 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:01:01.003774 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 9/18] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:01:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:01:01.004394 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b22702-5570-429b-a41f-e0c0402a57c7 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:01.008817 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bc551e-2f3d-4a08-a65f-20b1a4fe0e8a None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:01.029975 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:01.037394 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3a3efe-df66-44f5-864c-b9c2c9c0dc16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:01.038025 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3a3efe-df66-44f5-864c-b9c2c9c0dc16 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:01.043688 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3a3efe-df66-44f5-864c-b9c2c9c0dc16 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:01.043688 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3a3efe-df66-44f5-864c-b9c2c9c0dc16 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:01.043688 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3a3efe-df66-44f5-864c-b9c2c9c0dc16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:01.043688 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3a3efe-df66-44f5-864c-b9c2c9c0dc16 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:01.043688 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 10/19] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:01:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 00:01:01.048681 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3a3efe-df66-44f5-864c-b9c2c9c0dc16 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:01.109642 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bc551e-2f3d-4a08-a65f-20b1a4fe0e8a None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:01.127906 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:01.142864 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbff960-0346-4859-b676-4df7b021a0d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:01.143496 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbff960-0346-4859-b676-4df7b021a0d6 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:01.148517 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbff960-0346-4859-b676-4df7b021a0d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:01.148517 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbff960-0346-4859-b676-4df7b021a0d6 None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:01.148517 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbff960-0346-4859-b676-4df7b021a0d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:01:01.148517 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbff960-0346-4859-b676-4df7b021a0d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:01.148517 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbff960-0346-4859-b676-4df7b021a0d6 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:01.149074 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3a3efe-df66-44f5-864c-b9c2c9c0dc16 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:01.163300 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 10/20] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Oct 12 00:01:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 00:01:01.181568 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11141490-27a2-41d1-83b6-b3e9bc9f90d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:01.182258 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11141490-27a2-41d1-83b6-b3e9bc9f90d5 None admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:01.182531 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11141490-27a2-41d1-83b6-b3e9bc9f90d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:01.182807 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11141490-27a2-41d1-83b6-b3e9bc9f90d5 None admin] PATH_INFO: `/v3/domains/default/users/1fcb9022bfe743ae9405fc8470a5e016/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:01.184845 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11141490-27a2-41d1-83b6-b3e9bc9f90d5 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=1fcb9022bfe743ae9405fc8470a5e016, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:01.186338 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11141490-27a2-41d1-83b6-b3e9bc9f90d5 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:01.381467 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 11/21] 10.222.0.36 () {58 vars in 1361 bytes} [Wed Oct 12 00:01:01 2022] PUT /identity/v3/domains/default/users/1fcb9022bfe743ae9405fc8470a5e016/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 00:01:03.671202 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfed4c18-d522-4f6c-bfdf-32c590ddf293 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:03.672272 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfed4c18-d522-4f6c-bfdf-32c590ddf293 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:03.672761 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfed4c18-d522-4f6c-bfdf-32c590ddf293 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:03.674598 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 11/22] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:01:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:01:03.681147 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bce4dd5-c62a-45c1-b48f-80f4886b6598 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:03.682077 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bce4dd5-c62a-45c1-b48f-80f4886b6598 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:03.682567 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bce4dd5-c62a-45c1-b48f-80f4886b6598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:03.706340 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-1bce4dd5-c62a-45c1-b48f-80f4886b6598 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:01:03.708210 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-1bce4dd5-c62a-45c1-b48f-80f4886b6598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:01:03.799048 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 12/23] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:01:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 00:01:03.889797 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f1e133-f1ce-41be-b5e4-884d12765975 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:03.890411 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f1e133-f1ce-41be-b5e4-884d12765975 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:03.890924 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f1e133-f1ce-41be-b5e4-884d12765975 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:03.892471 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 12/24] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:01:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:01:03.902354 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dd6bb2-2388-46da-8477-c4ae954842d1 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:03.902879 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dd6bb2-2388-46da-8477-c4ae954842d1 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:03.903344 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dd6bb2-2388-46da-8477-c4ae954842d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:03.931532 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-d9dd6bb2-2388-46da-8477-c4ae954842d1 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:01:03.933115 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-d9dd6bb2-2388-46da-8477-c4ae954842d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:01:04.016377 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:04.021282 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 13/25] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:01:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 12 00:01:04.044017 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae45ec1-a4be-4eeb-aef1-4107c6166b08 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:04.044290 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae45ec1-a4be-4eeb-aef1-4107c6166b08 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:04.044767 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae45ec1-a4be-4eeb-aef1-4107c6166b08 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:04.046278 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 13/26] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:01:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:01:04.051641 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c017f-7d36-4595-ad3a-56c1f4019059 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:04.051877 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c017f-7d36-4595-ad3a-56c1f4019059 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:04.052159 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c017f-7d36-4595-ad3a-56c1f4019059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:04.077852 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-196c017f-7d36-4595-ad3a-56c1f4019059 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:01:04.079048 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-196c017f-7d36-4595-ad3a-56c1f4019059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:01:04.166705 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 14/27] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:01:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 12 00:01:04.299491 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae45ec1-a4be-4eeb-aef1-4107c6166b08 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:04.399483 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:04.426532 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae45ec1-a4be-4eeb-aef1-4107c6166b08 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:04.476285 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d8ce91-f81c-4519-801e-69bc761069c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:04.476960 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d8ce91-f81c-4519-801e-69bc761069c6 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:04.477238 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d8ce91-f81c-4519-801e-69bc761069c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:04.477510 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d8ce91-f81c-4519-801e-69bc761069c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:04.488070 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d8ce91-f81c-4519-801e-69bc761069c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:04.489540 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d8ce91-f81c-4519-801e-69bc761069c6 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:04.507786 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-29d8ce91-f81c-4519-801e-69bc761069c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:01:04.507786 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:01:04.507786 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:01:04.507786 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:01:04.507786 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:01:04.507786 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:01:04.507786 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:01:04.507786 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:01:04.507786 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:01:04.507786 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:01:04.507786 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:01:04.507786 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:01:04.507786 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 00:01:04.507786 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 00:01:04.507786 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 00:01:04.507786 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 00:01:04.509215 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:01:04.509215 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:01:04.509215 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:01:04.509215 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:01:04.509215 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:01:04.509215 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:01:04.509215 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:01:04.509215 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:01:04.509215 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:01:04.509215 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:01:04.509215 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:01:04.509215 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:01:04.509215 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:01:04.509215 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:01:04.509215 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:01:04.509215 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:01:04.509215 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 00:01:04.510620 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 00:01:04.510620 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:01:04.510620 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:01:04.510620 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 14/28] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:01:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:01:04.515659 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196c017f-7d36-4595-ad3a-56c1f4019059 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:04.541966 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d8ce91-f81c-4519-801e-69bc761069c6 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:04.674801 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196c017f-7d36-4595-ad3a-56c1f4019059 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:04.688636 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d8ce91-f81c-4519-801e-69bc761069c6 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:04.708535 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:04.714974 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c0d612-e95a-481b-8007-e9f7ad670833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:04.715806 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c0d612-e95a-481b-8007-e9f7ad670833 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:04.716229 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c0d612-e95a-481b-8007-e9f7ad670833 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:04.716705 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c0d612-e95a-481b-8007-e9f7ad670833 None admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:04.717536 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c0d612-e95a-481b-8007-e9f7ad670833 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:01:04.718009 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c0d612-e95a-481b-8007-e9f7ad670833 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:04.719804 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c0d612-e95a-481b-8007-e9f7ad670833 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:04.722652 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:04.728699 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746fe64f-941c-481d-bfb0-a0eb7da98ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:04.729454 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746fe64f-941c-481d-bfb0-a0eb7da98ca2 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:04.729806 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746fe64f-941c-481d-bfb0-a0eb7da98ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:04.730155 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746fe64f-941c-481d-bfb0-a0eb7da98ca2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:04.741526 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746fe64f-941c-481d-bfb0-a0eb7da98ca2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 00:01:04.742211 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 15/29] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Oct 12 00:01:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:04.744284 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8872b91d-34ee-4c3e-9150-ab0d48cdcd0d None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:04.744761 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8872b91d-34ee-4c3e-9150-ab0d48cdcd0d None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:04.745205 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8872b91d-34ee-4c3e-9150-ab0d48cdcd0d None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:04.746614 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 16/30] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:01:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:01:04.748145 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8872b91d-34ee-4c3e-9150-ab0d48cdcd0d None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:04.753799 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-746fe64f-941c-481d-bfb0-a0eb7da98ca2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 00:01:04.753799 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:01:04.753799 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:01:04.753799 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:01:04.753799 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:01:04.753799 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:01:04.753799 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:01:04.753799 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:01:04.753799 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:01:04.753799 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:01:04.753799 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:01:04.753799 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:01:04.753799 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 00:01:04.753799 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 00:01:04.753799 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 00:01:04.753799 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 00:01:04.756243 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 00:01:04.756243 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 00:01:04.756243 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 00:01:04.756243 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 00:01:04.756243 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:01:04.756243 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:01:04.756243 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:01:04.756243 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:01:04.756243 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:01:04.756243 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:01:04.756243 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:01:04.756243 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:01:04.756243 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:01:04.756243 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:01:04.756243 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:01:04.756243 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:01:04.756984 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:01:04.756984 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:01:04.756984 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:01:04.756984 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:01:04.756984 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 00:01:04.756984 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 00:01:04.756984 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 00:01:04.756984 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 00:01:04.756984 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 00:01:04.756984 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 00:01:04.756984 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 00:01:04.756984 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:01:04.756984 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 15/31] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Oct 12 00:01:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:01:04.760819 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746fe64f-941c-481d-bfb0-a0eb7da98ca2 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:04.866345 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8872b91d-34ee-4c3e-9150-ab0d48cdcd0d None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:04.878407 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746fe64f-941c-481d-bfb0-a0eb7da98ca2 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:04.906518 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4551717-1d8d-4cae-84fa-c48fd17a2be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:04.907423 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4551717-1d8d-4cae-84fa-c48fd17a2be6 None admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:04.907870 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4551717-1d8d-4cae-84fa-c48fd17a2be6 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:04.908397 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4551717-1d8d-4cae-84fa-c48fd17a2be6 None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:04.909354 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4551717-1d8d-4cae-84fa-c48fd17a2be6 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:04.910893 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4551717-1d8d-4cae-84fa-c48fd17a2be6 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:04.917696 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7884d935-8b6e-4b4a-82f3-c31934b4f8cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:04.918481 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7884d935-8b6e-4b4a-82f3-c31934b4f8cf None admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:04.918826 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7884d935-8b6e-4b4a-82f3-c31934b4f8cf None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:04.919164 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7884d935-8b6e-4b4a-82f3-c31934b4f8cf None admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:04.919968 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7884d935-8b6e-4b4a-82f3-c31934b4f8cf None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:04.921539 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7884d935-8b6e-4b4a-82f3-c31934b4f8cf None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:05.014317 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 17/32] 10.222.0.36 () {60 vars in 1131 bytes} [Wed Oct 12 00:01:04 2022] POST /identity/v3/roles => generated 225 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:01:05.015820 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c68b69c-b22e-43c6-91f4-7b39e54226ce None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:05.016266 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c68b69c-b22e-43c6-91f4-7b39e54226ce None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:05.016717 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c68b69c-b22e-43c6-91f4-7b39e54226ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:05.036586 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-2c68b69c-b22e-43c6-91f4-7b39e54226ce None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:01:05.038139 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-2c68b69c-b22e-43c6-91f4-7b39e54226ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:01:05.146121 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:05.193213 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 18/33] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:01:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 00:01:05.194362 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c68b69c-b22e-43c6-91f4-7b39e54226ce None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:05.271477 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 16/34] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Oct 12 00:01:04 2022] POST /identity/v3/projects => generated 301 bytes in 516 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:01:05.273113 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0045aadb-9efc-43cc-85c9-987d467d3dd6 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:05.273404 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0045aadb-9efc-43cc-85c9-987d467d3dd6 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:05.274255 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0045aadb-9efc-43cc-85c9-987d467d3dd6 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:05.280207 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 17/35] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:01:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:01:05.282341 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f57ec8-6aac-4d68-a23e-34da47090d18 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:05.282631 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f57ec8-6aac-4d68-a23e-34da47090d18 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:05.282912 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f57ec8-6aac-4d68-a23e-34da47090d18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:05.324571 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-37f57ec8-6aac-4d68-a23e-34da47090d18 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:01:05.326201 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-37f57ec8-6aac-4d68-a23e-34da47090d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:01:05.410910 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:05.446148 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 18/36] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:01:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 12 00:01:05.462863 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c68b69c-b22e-43c6-91f4-7b39e54226ce None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:05.502468 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae533797-f463-457a-aaf6-246a2ef09a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:05.503101 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae533797-f463-457a-aaf6-246a2ef09a0a None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:05.503394 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae533797-f463-457a-aaf6-246a2ef09a0a None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:05.503696 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae533797-f463-457a-aaf6-246a2ef09a0a None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:05.504380 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae533797-f463-457a-aaf6-246a2ef09a0a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:01:05.504767 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae533797-f463-457a-aaf6-246a2ef09a0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:05.506259 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae533797-f463-457a-aaf6-246a2ef09a0a None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:05.516856 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 19/37] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Oct 12 00:01:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:05.523101 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f57ec8-6aac-4d68-a23e-34da47090d18 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:05.596793 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae533797-f463-457a-aaf6-246a2ef09a0a None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:05.647915 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f57ec8-6aac-4d68-a23e-34da47090d18 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:05.652399 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:05.688539 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93277e9-e50d-43cf-aa60-50a788a60c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:05.689191 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93277e9-e50d-43cf-aa60-50a788a60c66 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:05.689736 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93277e9-e50d-43cf-aa60-50a788a60c66 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:05.689899 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93277e9-e50d-43cf-aa60-50a788a60c66 None admin] PATH_INFO: `/v3/users/1fcb9022bfe743ae9405fc8470a5e016` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:05.691964 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93277e9-e50d-43cf-aa60-50a788a60c66 None admin] RBAC: Authorizing `identity:get_user(user_id=1fcb9022bfe743ae9405fc8470a5e016)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:05.693569 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93277e9-e50d-43cf-aa60-50a788a60c66 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:05.695194 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 19/38] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Oct 12 00:01:05 2022] GET /identity/v3/users/1fcb9022bfe743ae9405fc8470a5e016 => generated 247 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:05.696630 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232b70f1-34ce-49e9-ac11-be1455d7f1fc None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:05.696911 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232b70f1-34ce-49e9-ac11-be1455d7f1fc None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:05.697181 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232b70f1-34ce-49e9-ac11-be1455d7f1fc None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:05.698528 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 20/39] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:01:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:01:05.703590 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232b70f1-34ce-49e9-ac11-be1455d7f1fc None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:05.733017 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae533797-f463-457a-aaf6-246a2ef09a0a None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:05.753286 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:05.768259 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92d1e81-3e01-4341-81f4-5c9ea9bdd155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:05.768882 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92d1e81-3e01-4341-81f4-5c9ea9bdd155 None admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:05.769161 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92d1e81-3e01-4341-81f4-5c9ea9bdd155 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:05.769434 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92d1e81-3e01-4341-81f4-5c9ea9bdd155 None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:05.770115 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92d1e81-3e01-4341-81f4-5c9ea9bdd155 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:05.771565 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92d1e81-3e01-4341-81f4-5c9ea9bdd155 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:05.811408 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232b70f1-34ce-49e9-ac11-be1455d7f1fc None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:05.844335 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78495fb4-0b1f-4849-86b6-525e1b1fc2b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:05.852333 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78495fb4-0b1f-4849-86b6-525e1b1fc2b2 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:05.852333 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78495fb4-0b1f-4849-86b6-525e1b1fc2b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:05.852333 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78495fb4-0b1f-4849-86b6-525e1b1fc2b2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:05.852333 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78495fb4-0b1f-4849-86b6-525e1b1fc2b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:05.852333 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78495fb4-0b1f-4849-86b6-525e1b1fc2b2 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:05.852333 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 21/40] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:01:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:05.852801 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f00dab-286e-4fbd-8d8a-90335f1562da None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:05.852801 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f00dab-286e-4fbd-8d8a-90335f1562da None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:05.852801 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f00dab-286e-4fbd-8d8a-90335f1562da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:05.868370 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-85f00dab-286e-4fbd-8d8a-90335f1562da None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:01:05.869554 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-85f00dab-286e-4fbd-8d8a-90335f1562da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:01:05.870225 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 20/41] 10.222.0.36 () {60 vars in 1131 bytes} [Wed Oct 12 00:01:05 2022] POST /identity/v3/roles => generated 219 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 00:01:05.871056 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92d1e81-3e01-4341-81f4-5c9ea9bdd155 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:05.947588 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 22/42] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:01:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 00:01:05.948469 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f00dab-286e-4fbd-8d8a-90335f1562da None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:05.997871 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92d1e81-3e01-4341-81f4-5c9ea9bdd155 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:05.999890 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:06.007837 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:06.044088 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908bc809-1f41-4f9f-ab72-eebb550934a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:06.044745 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908bc809-1f41-4f9f-ab72-eebb550934a8 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:06.045026 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908bc809-1f41-4f9f-ab72-eebb550934a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:06.045301 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908bc809-1f41-4f9f-ab72-eebb550934a8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:06.047160 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908bc809-1f41-4f9f-ab72-eebb550934a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:06.048740 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908bc809-1f41-4f9f-ab72-eebb550934a8 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:06.050604 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 21/43] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:01:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:06.056431 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908bc809-1f41-4f9f-ab72-eebb550934a8 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:06.084253 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f00dab-286e-4fbd-8d8a-90335f1562da None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:06.124118 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8861a850-5fb2-4581-8891-03dd36358ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:06.124752 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8861a850-5fb2-4581-8891-03dd36358ebd None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:06.125037 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8861a850-5fb2-4581-8891-03dd36358ebd None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:06.125313 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8861a850-5fb2-4581-8891-03dd36358ebd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:06.126045 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8861a850-5fb2-4581-8891-03dd36358ebd None admin] RBAC: Adding query filter params (role.id=b2e41f0d7a304dc7af8b6a232049b6a2, scope.domain.id=default, user.id=1fcb9022bfe743ae9405fc8470a5e016) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:01:06.126353 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8861a850-5fb2-4581-8891-03dd36358ebd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:06.128314 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8861a850-5fb2-4581-8891-03dd36358ebd None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:06.137379 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 23/44] 10.222.0.36 () {56 vars in 1322 bytes} [Wed Oct 12 00:01:05 2022] GET /identity/v3/role_assignments?user.id=1fcb9022bfe743ae9405fc8470a5e016&scope.domain.id=default&role.id=b2e41f0d7a304dc7af8b6a232049b6a2 => generated 530 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:06.183004 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908bc809-1f41-4f9f-ab72-eebb550934a8 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:06.221835 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8861a850-5fb2-4581-8891-03dd36358ebd None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:06.222838 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ec8ca5-700a-4611-a63d-bda7b7329023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:06.223518 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ec8ca5-700a-4611-a63d-bda7b7329023 None admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:06.223803 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ec8ca5-700a-4611-a63d-bda7b7329023 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:06.224074 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ec8ca5-700a-4611-a63d-bda7b7329023 None admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:06.224746 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ec8ca5-700a-4611-a63d-bda7b7329023 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:06.226274 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ec8ca5-700a-4611-a63d-bda7b7329023 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:06.313616 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 22/45] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Oct 12 00:01:06 2022] POST /identity/v3/projects => generated 312 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:01:06.334832 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8861a850-5fb2-4581-8891-03dd36358ebd None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:06.338033 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:06.373993 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b4d9be-43b1-4bd5-b8cc-f95df0db03ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:06.374635 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b4d9be-43b1-4bd5-b8cc-f95df0db03ae None admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:06.374938 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b4d9be-43b1-4bd5-b8cc-f95df0db03ae None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:06.375213 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b4d9be-43b1-4bd5-b8cc-f95df0db03ae None admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:06.375980 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b4d9be-43b1-4bd5-b8cc-f95df0db03ae None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:06.377489 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b4d9be-43b1-4bd5-b8cc-f95df0db03ae None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:06.446562 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 24/46] 10.222.0.36 () {60 vars in 1131 bytes} [Wed Oct 12 00:01:06 2022] POST /identity/v3/roles => generated 223 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 00:01:08.460125 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7601ce52-8511-41f4-ba05-f9cb15da6a86 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:08.460640 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7601ce52-8511-41f4-ba05-f9cb15da6a86 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:08.460752 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7601ce52-8511-41f4-ba05-f9cb15da6a86 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:08.462060 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 23/47] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:01:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:01:08.467176 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf7cbcd-1380-49e2-82d3-9f26f11fc5aa None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:08.467571 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf7cbcd-1380-49e2-82d3-9f26f11fc5aa None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:08.467763 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf7cbcd-1380-49e2-82d3-9f26f11fc5aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:08.485996 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-cdf7cbcd-1380-49e2-82d3-9f26f11fc5aa None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:01:08.487198 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-cdf7cbcd-1380-49e2-82d3-9f26f11fc5aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:01:08.553016 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 25/48] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:01:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 00:01:08.823576 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7601ce52-8511-41f4-ba05-f9cb15da6a86 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:08.848385 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:08.925350 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7601ce52-8511-41f4-ba05-f9cb15da6a86 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:08.957190 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae191d25-ea88-4fb2-a9e7-83f25f5dabe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:08.957817 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae191d25-ea88-4fb2-a9e7-83f25f5dabe3 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:08.958098 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae191d25-ea88-4fb2-a9e7-83f25f5dabe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:08.958371 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae191d25-ea88-4fb2-a9e7-83f25f5dabe3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:08.959517 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae191d25-ea88-4fb2-a9e7-83f25f5dabe3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:08.961467 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae191d25-ea88-4fb2-a9e7-83f25f5dabe3 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:08.963017 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 24/49] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:01:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:08.968560 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf7cbcd-1380-49e2-82d3-9f26f11fc5aa None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:09.058426 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:09.064903 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf7cbcd-1380-49e2-82d3-9f26f11fc5aa None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:09.098423 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3506514a-29f8-4ad2-8f72-e63e2acb4513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:09.098974 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506514a-29f8-4ad2-8f72-e63e2acb4513 None admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:09.099253 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506514a-29f8-4ad2-8f72-e63e2acb4513 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:09.099554 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506514a-29f8-4ad2-8f72-e63e2acb4513 None admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:09.100207 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3506514a-29f8-4ad2-8f72-e63e2acb4513 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:09.102239 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3506514a-29f8-4ad2-8f72-e63e2acb4513 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:09.184228 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 26/50] 10.222.0.36 () {60 vars in 1140 bytes} [Wed Oct 12 00:01:08 2022] POST /identity/v3/projects => generated 298 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:01:11.316138 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f3fb6-65ae-4e99-b9d9-7d1c8f875240 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:11.316686 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f3fb6-65ae-4e99-b9d9-7d1c8f875240 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:11.316686 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f3fb6-65ae-4e99-b9d9-7d1c8f875240 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:11.318018 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 25/51] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:01:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:01:11.323943 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6997c7-e85b-40a0-8e60-f9e8361f7d5d None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:11.324313 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6997c7-e85b-40a0-8e60-f9e8361f7d5d None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:11.324502 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6997c7-e85b-40a0-8e60-f9e8361f7d5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:11.444789 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-8b6997c7-e85b-40a0-8e60-f9e8361f7d5d None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:01:11.446570 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-8b6997c7-e85b-40a0-8e60-f9e8361f7d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:01:11.510515 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 27/52] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:01:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 00:01:11.782854 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351f3fb6-65ae-4e99-b9d9-7d1c8f875240 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:11.870270 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:11.880104 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351f3fb6-65ae-4e99-b9d9-7d1c8f875240 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:11.912024 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15e616a-bc90-4de6-8def-814745ab131d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:11.912643 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e616a-bc90-4de6-8def-814745ab131d None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:11.912920 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e616a-bc90-4de6-8def-814745ab131d None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:11.913190 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e616a-bc90-4de6-8def-814745ab131d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:11.914309 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15e616a-bc90-4de6-8def-814745ab131d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:11.915774 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15e616a-bc90-4de6-8def-814745ab131d None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:11.917306 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 26/53] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:01:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:11.922721 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6997c7-e85b-40a0-8e60-f9e8361f7d5d None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:11.996535 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:12.019095 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6997c7-e85b-40a0-8e60-f9e8361f7d5d None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:12.051430 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27bbbca-a992-4d26-93b0-e8f4bd0a2211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:12.051987 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27bbbca-a992-4d26-93b0-e8f4bd0a2211 None admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:12.052266 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27bbbca-a992-4d26-93b0-e8f4bd0a2211 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:12.052540 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27bbbca-a992-4d26-93b0-e8f4bd0a2211 None admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:12.053182 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27bbbca-a992-4d26-93b0-e8f4bd0a2211 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:12.055183 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27bbbca-a992-4d26-93b0-e8f4bd0a2211 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:12.148756 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 28/54] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Oct 12 00:01:11 2022] POST /identity/v3/users => generated 275 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:01:15.232053 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06616ff-5134-422d-b85d-b407a534c049 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:15.233081 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06616ff-5134-422d-b85d-b407a534c049 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:15.233563 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06616ff-5134-422d-b85d-b407a534c049 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:15.235106 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 27/55] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:01:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:01:15.240479 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eb12d9-d4c5-41b0-9c0b-1d3e4cce34fb None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:15.241269 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eb12d9-d4c5-41b0-9c0b-1d3e4cce34fb None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:15.242292 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eb12d9-d4c5-41b0-9c0b-1d3e4cce34fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:15.272406 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-33eb12d9-d4c5-41b0-9c0b-1d3e4cce34fb None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:01:15.273975 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-33eb12d9-d4c5-41b0-9c0b-1d3e4cce34fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:01:15.364947 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 29/56] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:01:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 00:01:15.442210 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c9b9c-d473-4f44-9e81-c21fcf0a1d97 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:15.442823 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c9b9c-d473-4f44-9e81-c21fcf0a1d97 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:15.443309 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c9b9c-d473-4f44-9e81-c21fcf0a1d97 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:15.444934 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 28/57] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:01:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:01:15.456498 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f5202f-324b-422e-b4b2-ed0bae5d7272 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:15.457007 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f5202f-324b-422e-b4b2-ed0bae5d7272 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:15.457465 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f5202f-324b-422e-b4b2-ed0bae5d7272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:15.488463 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-64f5202f-324b-422e-b4b2-ed0bae5d7272 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:01:15.489978 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-64f5202f-324b-422e-b4b2-ed0bae5d7272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:01:15.574743 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:15.592699 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 30/58] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:01:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 00:01:15.647161 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f63ab9-10b5-47ed-a205-1bb5d8fb5889 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:15.647460 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f63ab9-10b5-47ed-a205-1bb5d8fb5889 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:15.648482 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f63ab9-10b5-47ed-a205-1bb5d8fb5889 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:15.650044 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 29/59] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:01:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:01:15.655384 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c18d071-5b1d-4e72-a9cb-460ad9681efb None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:15.655680 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c18d071-5b1d-4e72-a9cb-460ad9681efb None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:15.655956 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c18d071-5b1d-4e72-a9cb-460ad9681efb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:15.675404 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-0c18d071-5b1d-4e72-a9cb-460ad9681efb None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:01:15.676612 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-0c18d071-5b1d-4e72-a9cb-460ad9681efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:01:15.770002 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f63ab9-10b5-47ed-a205-1bb5d8fb5889 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:15.774878 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 31/60] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:01:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 00:01:15.906641 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f63ab9-10b5-47ed-a205-1bb5d8fb5889 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:15.942396 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b14eb-afd1-411d-8760-c961937e727a None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:15.942968 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b14eb-afd1-411d-8760-c961937e727a None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:15.943517 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b14eb-afd1-411d-8760-c961937e727a None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:15.945901 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 32/61] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:01:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:01:15.948511 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19859e01-704c-4b3c-85cf-27f76d3a9242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:15.950271 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b1bce-a920-4c20-8307-01f8e852281e None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:15.950743 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b1bce-a920-4c20-8307-01f8e852281e None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:15.951180 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b1bce-a920-4c20-8307-01f8e852281e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:15.957715 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19859e01-704c-4b3c-85cf-27f76d3a9242 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:15.959149 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19859e01-704c-4b3c-85cf-27f76d3a9242 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:15.959628 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19859e01-704c-4b3c-85cf-27f76d3a9242 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:15.979762 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-ae1b1bce-a920-4c20-8307-01f8e852281e None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:01:15.983065 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-ae1b1bce-a920-4c20-8307-01f8e852281e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:01:15.985836 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19859e01-704c-4b3c-85cf-27f76d3a9242 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 00:01:15.995832 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-19859e01-704c-4b3c-85cf-27f76d3a9242 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 12 00:01:15.995832 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:01:15.995832 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:01:15.995832 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:01:15.995832 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:01:15.995832 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:01:15.995832 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:01:15.995832 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:01:15.995832 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:01:15.995832 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:01:15.995832 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:01:15.995832 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:01:15.995832 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 00:01:15.995832 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 00:01:15.995832 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 00:01:15.995832 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 00:01:15.996715 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 00:01:15.996715 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 00:01:15.996715 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 00:01:15.996715 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 00:01:15.996715 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:01:15.996715 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:01:15.996715 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:01:15.996715 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:01:15.996715 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:01:15.996715 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:01:15.996715 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:01:15.996715 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:01:15.996715 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:01:15.996715 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:01:15.996715 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:01:15.996715 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:01:15.997462 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:01:15.997462 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:01:15.997462 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:01:15.997462 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:01:15.997462 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 00:01:15.997462 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 00:01:15.997462 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 00:01:15.997462 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 00:01:15.997462 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 00:01:15.997462 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 00:01:15.997462 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 12 00:01:15.997462 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:01:15.999690 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 30/62] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Oct 12 00:01:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 00:01:16.005551 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19859e01-704c-4b3c-85cf-27f76d3a9242 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:16.035406 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:16.083198 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 33/63] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:01:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 00:01:16.083881 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1b1bce-a920-4c20-8307-01f8e852281e None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:16.139051 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19859e01-704c-4b3c-85cf-27f76d3a9242 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:16.143818 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:16.192308 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e14807b-7fb5-4580-b2a9-8554fcc190aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:16.193325 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14807b-7fb5-4580-b2a9-8554fcc190aa None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:16.193746 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14807b-7fb5-4580-b2a9-8554fcc190aa None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:16.194456 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14807b-7fb5-4580-b2a9-8554fcc190aa None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:16.195712 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e14807b-7fb5-4580-b2a9-8554fcc190aa None admin] RBAC: Adding query filter params (name=member) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:01:16.196173 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e14807b-7fb5-4580-b2a9-8554fcc190aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:16.197703 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e14807b-7fb5-4580-b2a9-8554fcc190aa None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:16.198352 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1b1bce-a920-4c20-8307-01f8e852281e None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:16.207964 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 31/64] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Oct 12 00:01:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:16.209737 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e14807b-7fb5-4580-b2a9-8554fcc190aa None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:16.246876 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb113d53-3217-4b87-ac9b-24f5d6642609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:16.248363 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb113d53-3217-4b87-ac9b-24f5d6642609 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:16.248785 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb113d53-3217-4b87-ac9b-24f5d6642609 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:16.250089 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb113d53-3217-4b87-ac9b-24f5d6642609 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:16.265267 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb113d53-3217-4b87-ac9b-24f5d6642609 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 00:01:16.271641 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-bb113d53-3217-4b87-ac9b-24f5d6642609 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 12 00:01:16.271641 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:01:16.271641 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:01:16.271641 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:01:16.271641 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:01:16.271641 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:01:16.271641 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:01:16.271641 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:01:16.271641 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:01:16.271641 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:01:16.271641 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:01:16.271641 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:01:16.271641 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 00:01:16.271641 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 00:01:16.271641 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 00:01:16.271641 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 00:01:16.272494 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 00:01:16.272494 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 00:01:16.272494 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 00:01:16.272494 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 00:01:16.272494 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:01:16.272494 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:01:16.272494 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:01:16.272494 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:01:16.272494 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:01:16.272494 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:01:16.272494 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:01:16.272494 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:01:16.272494 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:01:16.272494 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:01:16.272494 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:01:16.272494 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:01:16.273279 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:01:16.273279 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:01:16.273279 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:01:16.273279 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:01:16.273279 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 00:01:16.273279 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 00:01:16.273279 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 00:01:16.273279 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 00:01:16.273279 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 00:01:16.273279 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 00:01:16.273279 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 12 00:01:16.273279 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:01:16.273279 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 34/65] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Oct 12 00:01:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:01:16.274016 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb113d53-3217-4b87-ac9b-24f5d6642609 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:16.304856 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:16.323280 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e14807b-7fb5-4580-b2a9-8554fcc190aa None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:16.360308 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90c9189-6dd3-4b97-a68a-2238cee544ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:16.360949 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90c9189-6dd3-4b97-a68a-2238cee544ba None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:16.361226 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90c9189-6dd3-4b97-a68a-2238cee544ba None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:16.361499 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90c9189-6dd3-4b97-a68a-2238cee544ba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:16.371357 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90c9189-6dd3-4b97-a68a-2238cee544ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 00:01:16.376223 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-b90c9189-6dd3-4b97-a68a-2238cee544ba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 00:01:16.376223 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:01:16.376223 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:01:16.376223 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:01:16.376223 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:01:16.376223 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:01:16.376223 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:01:16.376223 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:01:16.376223 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:01:16.376223 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:01:16.376223 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:01:16.376223 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:01:16.376223 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 00:01:16.376223 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 00:01:16.376223 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 00:01:16.376223 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 00:01:16.377036 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 00:01:16.377036 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 00:01:16.377036 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 00:01:16.377036 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 00:01:16.377036 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:01:16.377036 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:01:16.377036 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:01:16.377036 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:01:16.377036 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:01:16.377036 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:01:16.377036 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:01:16.377036 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:01:16.377036 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:01:16.377036 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:01:16.377036 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:01:16.377036 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:01:16.377814 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:01:16.377814 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:01:16.377814 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:01:16.377814 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:01:16.377814 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 00:01:16.377814 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 00:01:16.377814 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 00:01:16.377814 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 00:01:16.377814 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 00:01:16.377814 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 00:01:16.377814 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 00:01:16.377814 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:01:16.377814 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 32/66] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Oct 12 00:01:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:01:16.378462 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee83470-eabb-4a38-bda3-ea9c9211ab80 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:16.378729 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee83470-eabb-4a38-bda3-ea9c9211ab80 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:16.378994 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee83470-eabb-4a38-bda3-ea9c9211ab80 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:16.380391 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 33/67] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:01:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:01:16.380920 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee83470-eabb-4a38-bda3-ea9c9211ab80 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:16.383410 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb113d53-3217-4b87-ac9b-24f5d6642609 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:16.398796 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:16.421262 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e33639-c035-4475-bc9c-ef93a5e166f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:16.421909 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e33639-c035-4475-bc9c-ef93a5e166f6 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:16.422192 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e33639-c035-4475-bc9c-ef93a5e166f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:16.422470 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e33639-c035-4475-bc9c-ef93a5e166f6 None admin] PATH_INFO: `/v3/users/216d4237a0a44e7eaf170e6dcd700023` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:16.438134 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e33639-c035-4475-bc9c-ef93a5e166f6 None admin] RBAC: Authorizing `identity:get_user(user_id=216d4237a0a44e7eaf170e6dcd700023)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:16.439641 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e33639-c035-4475-bc9c-ef93a5e166f6 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:16.441250 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 35/68] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Oct 12 00:01:16 2022] GET /identity/v3/users/216d4237a0a44e7eaf170e6dcd700023 => generated 275 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:16.442012 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e33639-c035-4475-bc9c-ef93a5e166f6 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:16.484250 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee83470-eabb-4a38-bda3-ea9c9211ab80 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:16.516152 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8728f486-240b-4da8-9a2d-8afef525fa28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:16.516758 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8728f486-240b-4da8-9a2d-8afef525fa28 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:16.517034 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8728f486-240b-4da8-9a2d-8afef525fa28 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:16.517303 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8728f486-240b-4da8-9a2d-8afef525fa28 None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:16.517957 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8728f486-240b-4da8-9a2d-8afef525fa28 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:01:16.518257 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8728f486-240b-4da8-9a2d-8afef525fa28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:16.519658 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8728f486-240b-4da8-9a2d-8afef525fa28 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:16.527805 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 34/69] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Oct 12 00:01:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:16.528571 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8728f486-240b-4da8-9a2d-8afef525fa28 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:16.530952 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:16.545103 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e33639-c035-4475-bc9c-ef93a5e166f6 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:16.580279 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c971cb-e5d6-48d0-ad26-04a0b32579dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:16.580918 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c971cb-e5d6-48d0-ad26-04a0b32579dc None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:16.581193 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c971cb-e5d6-48d0-ad26-04a0b32579dc None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:16.581464 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c971cb-e5d6-48d0-ad26-04a0b32579dc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:16.591713 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c971cb-e5d6-48d0-ad26-04a0b32579dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 00:01:16.596926 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-c9c971cb-e5d6-48d0-ad26-04a0b32579dc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 12 00:01:16.596926 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:01:16.596926 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:01:16.596926 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:01:16.596926 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:01:16.596926 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:01:16.596926 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:01:16.596926 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:01:16.596926 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:01:16.596926 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:01:16.596926 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:01:16.596926 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:01:16.596926 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 00:01:16.596926 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 00:01:16.596926 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 00:01:16.596926 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 00:01:16.597777 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 00:01:16.597777 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 00:01:16.597777 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 00:01:16.597777 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 00:01:16.597777 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:01:16.597777 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:01:16.597777 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:01:16.597777 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:01:16.597777 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:01:16.597777 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:01:16.597777 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:01:16.597777 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:01:16.597777 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:01:16.597777 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:01:16.597777 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:01:16.597777 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:01:16.598786 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:01:16.598786 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:01:16.598786 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:01:16.598786 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:01:16.598786 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 00:01:16.598786 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 00:01:16.598786 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 00:01:16.598786 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 00:01:16.598786 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 00:01:16.598786 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 00:01:16.598786 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 12 00:01:16.598786 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:01:16.598786 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 36/70] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Oct 12 00:01:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:01:16.600564 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2bbf1-d4af-4b3d-9f60-5fa77d8ba921 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:16.601559 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2bbf1-d4af-4b3d-9f60-5fa77d8ba921 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:16.601838 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2bbf1-d4af-4b3d-9f60-5fa77d8ba921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:16.619521 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-1bf2bbf1-d4af-4b3d-9f60-5fa77d8ba921 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:01:16.620723 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-1bf2bbf1-d4af-4b3d-9f60-5fa77d8ba921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:01:16.627286 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8728f486-240b-4da8-9a2d-8afef525fa28 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:16.645218 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:16.660607 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097dd067-b813-4821-af87-11f5f151424d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:16.661215 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097dd067-b813-4821-af87-11f5f151424d None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:16.661494 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097dd067-b813-4821-af87-11f5f151424d None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:16.661769 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097dd067-b813-4821-af87-11f5f151424d None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:16.662427 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097dd067-b813-4821-af87-11f5f151424d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:01:16.662728 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097dd067-b813-4821-af87-11f5f151424d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:16.664669 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097dd067-b813-4821-af87-11f5f151424d None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:16.672493 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 35/71] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Oct 12 00:01:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:16.673266 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097dd067-b813-4821-af87-11f5f151424d None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:16.689111 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 37/72] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:01:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 00:01:16.690218 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf2bbf1-d4af-4b3d-9f60-5fa77d8ba921 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:16.736742 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:16.779501 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097dd067-b813-4821-af87-11f5f151424d None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:16.794371 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf2bbf1-d4af-4b3d-9f60-5fa77d8ba921 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:16.814753 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aec84c-6875-414e-928c-a2f03661e039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:16.815428 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aec84c-6875-414e-928c-a2f03661e039 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:16.815718 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aec84c-6875-414e-928c-a2f03661e039 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:16.815995 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aec84c-6875-414e-928c-a2f03661e039 None admin] PATH_INFO: `/v3/projects/410027d6c764410ea5b76db0171903df` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:16.817148 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aec84c-6875-414e-928c-a2f03661e039 None admin] RBAC: Authorizing `identity:get_project(project_id=410027d6c764410ea5b76db0171903df)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:16.818815 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aec84c-6875-414e-928c-a2f03661e039 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:16.820957 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 36/73] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Oct 12 00:01:16 2022] GET /identity/v3/projects/410027d6c764410ea5b76db0171903df => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:16.826316 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aec84c-6875-414e-928c-a2f03661e039 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:16.831215 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e5f3c7-b7b1-48eb-9708-97df65664581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:16.831831 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e5f3c7-b7b1-48eb-9708-97df65664581 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:16.832114 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e5f3c7-b7b1-48eb-9708-97df65664581 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:16.832391 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e5f3c7-b7b1-48eb-9708-97df65664581 None admin] PATH_INFO: `/v3/users/216d4237a0a44e7eaf170e6dcd700023` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:16.833532 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e5f3c7-b7b1-48eb-9708-97df65664581 None admin] RBAC: Authorizing `identity:get_user(user_id=216d4237a0a44e7eaf170e6dcd700023)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:16.836027 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e5f3c7-b7b1-48eb-9708-97df65664581 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:16.842738 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 38/74] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Oct 12 00:01:16 2022] GET /identity/v3/users/216d4237a0a44e7eaf170e6dcd700023 => generated 275 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:16.842738 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e5f3c7-b7b1-48eb-9708-97df65664581 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:16.871777 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:16.930369 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aec84c-6875-414e-928c-a2f03661e039 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:16.950028 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e5f3c7-b7b1-48eb-9708-97df65664581 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:16.962271 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:16.967218 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dfff0e-ccd9-4c69-9a1c-c83935eb208b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:16.967863 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dfff0e-ccd9-4c69-9a1c-c83935eb208b None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:16.968146 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dfff0e-ccd9-4c69-9a1c-c83935eb208b None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:16.968477 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dfff0e-ccd9-4c69-9a1c-c83935eb208b None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:16.969146 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dfff0e-ccd9-4c69-9a1c-c83935eb208b None admin] RBAC: Adding query filter params (name=member) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:01:16.975143 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dfff0e-ccd9-4c69-9a1c-c83935eb208b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:16.975143 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dfff0e-ccd9-4c69-9a1c-c83935eb208b None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:16.983566 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 37/75] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Oct 12 00:01:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:16.983566 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dfff0e-ccd9-4c69-9a1c-c83935eb208b None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:16.987860 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5455d782-cf37-4227-bb2c-fed5b3ab4ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:16.988498 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5455d782-cf37-4227-bb2c-fed5b3ab4ba9 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:16.988780 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5455d782-cf37-4227-bb2c-fed5b3ab4ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:16.989059 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5455d782-cf37-4227-bb2c-fed5b3ab4ba9 None admin] PATH_INFO: `/v3/users/1fcb9022bfe743ae9405fc8470a5e016` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:16.990203 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5455d782-cf37-4227-bb2c-fed5b3ab4ba9 None admin] RBAC: Authorizing `identity:get_user(user_id=1fcb9022bfe743ae9405fc8470a5e016)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:16.992277 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5455d782-cf37-4227-bb2c-fed5b3ab4ba9 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:16.993859 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 39/76] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Oct 12 00:01:16 2022] GET /identity/v3/users/1fcb9022bfe743ae9405fc8470a5e016 => generated 247 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:16.994616 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5455d782-cf37-4227-bb2c-fed5b3ab4ba9 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:17.077540 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dfff0e-ccd9-4c69-9a1c-c83935eb208b None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:17.091850 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:17.097759 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5455d782-cf37-4227-bb2c-fed5b3ab4ba9 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:17.110567 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40b3727-99ad-4f44-9fc5-55c3bc6cf426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:17.111229 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40b3727-99ad-4f44-9fc5-55c3bc6cf426 None admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:17.111541 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40b3727-99ad-4f44-9fc5-55c3bc6cf426 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:17.111821 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40b3727-99ad-4f44-9fc5-55c3bc6cf426 None admin] PATH_INFO: `/v3/projects/410027d6c764410ea5b76db0171903df/users/216d4237a0a44e7eaf170e6dcd700023/roles/ef689f96442945e8a7bc8cbfee13d909` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:17.113721 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40b3727-99ad-4f44-9fc5-55c3bc6cf426 None admin] RBAC: Authorizing `identity:create_grant(project_id=410027d6c764410ea5b76db0171903df, user_id=216d4237a0a44e7eaf170e6dcd700023, role_id=ef689f96442945e8a7bc8cbfee13d909)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:17.115162 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40b3727-99ad-4f44-9fc5-55c3bc6cf426 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:17.131923 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa40fa3f-5587-4fd7-907d-8a9e92df3bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:17.132577 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa40fa3f-5587-4fd7-907d-8a9e92df3bd7 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:17.132858 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa40fa3f-5587-4fd7-907d-8a9e92df3bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:17.133130 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa40fa3f-5587-4fd7-907d-8a9e92df3bd7 None admin] PATH_INFO: `/v3/projects/410027d6c764410ea5b76db0171903df` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:17.134251 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa40fa3f-5587-4fd7-907d-8a9e92df3bd7 None admin] RBAC: Authorizing `identity:get_project(project_id=410027d6c764410ea5b76db0171903df)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:17.135990 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa40fa3f-5587-4fd7-907d-8a9e92df3bd7 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:17.137614 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 40/77] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Oct 12 00:01:16 2022] GET /identity/v3/projects/410027d6c764410ea5b76db0171903df => generated 298 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:17.138366 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa40fa3f-5587-4fd7-907d-8a9e92df3bd7 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:17.175144 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 38/78] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Oct 12 00:01:16 2022] PUT /identity/v3/projects/410027d6c764410ea5b76db0171903df/users/216d4237a0a44e7eaf170e6dcd700023/roles/ef689f96442945e8a7bc8cbfee13d909 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:01:17.183435 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40b3727-99ad-4f44-9fc5-55c3bc6cf426 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:17.240585 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:17.259929 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa40fa3f-5587-4fd7-907d-8a9e92df3bd7 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:17.293025 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40b3727-99ad-4f44-9fc5-55c3bc6cf426 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:17.301126 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cad415-c4fb-4727-8f03-a7e48b6c9ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:17.301782 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cad415-c4fb-4727-8f03-a7e48b6c9ef3 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:17.302067 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cad415-c4fb-4727-8f03-a7e48b6c9ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:17.302344 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cad415-c4fb-4727-8f03-a7e48b6c9ef3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:17.303589 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cad415-c4fb-4727-8f03-a7e48b6c9ef3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:17.305641 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cad415-c4fb-4727-8f03-a7e48b6c9ef3 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:17.307338 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 41/79] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:01:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:17.308168 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cad415-c4fb-4727-8f03-a7e48b6c9ef3 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:17.331498 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47605253-cc30-4394-ae8e-8b5038f98249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:17.332175 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47605253-cc30-4394-ae8e-8b5038f98249 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:17.332431 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47605253-cc30-4394-ae8e-8b5038f98249 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:17.332706 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47605253-cc30-4394-ae8e-8b5038f98249 None admin] PATH_INFO: `/v3/users/216d4237a0a44e7eaf170e6dcd700023` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:17.333916 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47605253-cc30-4394-ae8e-8b5038f98249 None admin] RBAC: Authorizing `identity:get_user(user_id=216d4237a0a44e7eaf170e6dcd700023)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:17.335498 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47605253-cc30-4394-ae8e-8b5038f98249 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:17.341966 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 39/80] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Oct 12 00:01:17 2022] GET /identity/v3/users/216d4237a0a44e7eaf170e6dcd700023 => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:17.341966 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47605253-cc30-4394-ae8e-8b5038f98249 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:17.355944 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:17.432253 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cad415-c4fb-4727-8f03-a7e48b6c9ef3 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:17.445349 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47605253-cc30-4394-ae8e-8b5038f98249 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:17.467260 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:17.469873 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00374be5-5555-41e2-a6dc-82c28c19741e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:17.470520 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00374be5-5555-41e2-a6dc-82c28c19741e None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:17.470795 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00374be5-5555-41e2-a6dc-82c28c19741e None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:17.471071 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00374be5-5555-41e2-a6dc-82c28c19741e None admin] PATH_INFO: `/v3/projects/410027d6c764410ea5b76db0171903df` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:17.472351 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00374be5-5555-41e2-a6dc-82c28c19741e None admin] RBAC: Authorizing `identity:get_project(project_id=410027d6c764410ea5b76db0171903df)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:17.473793 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00374be5-5555-41e2-a6dc-82c28c19741e None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:17.475616 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 42/81] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Oct 12 00:01:17 2022] GET /identity/v3/projects/410027d6c764410ea5b76db0171903df => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:17.480532 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f727e962-d0a8-4a8b-bf08-577a1ada314a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:17.480662 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00374be5-5555-41e2-a6dc-82c28c19741e None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:17.480759 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f727e962-d0a8-4a8b-bf08-577a1ada314a None admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:17.480759 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f727e962-d0a8-4a8b-bf08-577a1ada314a None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:17.480898 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f727e962-d0a8-4a8b-bf08-577a1ada314a None admin] PATH_INFO: `/v3/projects/410027d6c764410ea5b76db0171903df/users/216d4237a0a44e7eaf170e6dcd700023/roles/81991615ac0b46818426bf37015df930` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:17.483118 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f727e962-d0a8-4a8b-bf08-577a1ada314a None admin] RBAC: Authorizing `identity:create_grant(project_id=410027d6c764410ea5b76db0171903df, user_id=216d4237a0a44e7eaf170e6dcd700023, role_id=81991615ac0b46818426bf37015df930)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:17.484589 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f727e962-d0a8-4a8b-bf08-577a1ada314a None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:17.569899 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 40/82] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Oct 12 00:01:17 2022] PUT /identity/v3/projects/410027d6c764410ea5b76db0171903df/users/216d4237a0a44e7eaf170e6dcd700023/roles/81991615ac0b46818426bf37015df930 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:01:17.581259 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f727e962-d0a8-4a8b-bf08-577a1ada314a None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:17.585695 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:17.589107 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00374be5-5555-41e2-a6dc-82c28c19741e None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:17.643453 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b9f2e6-a388-4196-b8b2-3362949f3ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:17.644355 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b9f2e6-a388-4196-b8b2-3362949f3ef2 None admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:17.645271 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b9f2e6-a388-4196-b8b2-3362949f3ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:17.645724 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b9f2e6-a388-4196-b8b2-3362949f3ef2 None admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:17.646585 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b9f2e6-a388-4196-b8b2-3362949f3ef2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:17.649045 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b9f2e6-a388-4196-b8b2-3362949f3ef2 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:17.719013 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f727e962-d0a8-4a8b-bf08-577a1ada314a None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:17.758373 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832097c0-0899-4b53-b715-ef45ffc0afa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:17.759017 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832097c0-0899-4b53-b715-ef45ffc0afa6 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:17.759297 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832097c0-0899-4b53-b715-ef45ffc0afa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:17.759640 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832097c0-0899-4b53-b715-ef45ffc0afa6 None admin] PATH_INFO: `/v3/projects/463d7341c0f949bdb062115f14db583f` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:17.761035 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832097c0-0899-4b53-b715-ef45ffc0afa6 None admin] RBAC: Authorizing `identity:get_project(project_id=463d7341c0f949bdb062115f14db583f)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:17.762524 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832097c0-0899-4b53-b715-ef45ffc0afa6 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:17.764243 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 41/83] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Oct 12 00:01:17 2022] GET /identity/v3/projects/463d7341c0f949bdb062115f14db583f => generated 312 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:17.765068 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832097c0-0899-4b53-b715-ef45ffc0afa6 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:17.766723 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 43/84] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Oct 12 00:01:17 2022] POST /identity/v3/users => generated 289 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:01:17.771242 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b9f2e6-a388-4196-b8b2-3362949f3ef2 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:17.837833 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:17.952050 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b9f2e6-a388-4196-b8b2-3362949f3ef2 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:17.967598 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832097c0-0899-4b53-b715-ef45ffc0afa6 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:18.003189 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f694afeb-90b3-4582-b8b3-e0833120dbe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:18.004460 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f694afeb-90b3-4582-b8b3-e0833120dbe5 None admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:18.004809 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f694afeb-90b3-4582-b8b3-e0833120dbe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:18.005096 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f694afeb-90b3-4582-b8b3-e0833120dbe5 None admin] PATH_INFO: `/v3/projects/463d7341c0f949bdb062115f14db583f/users/216d4237a0a44e7eaf170e6dcd700023/roles/ef689f96442945e8a7bc8cbfee13d909` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:18.008032 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f694afeb-90b3-4582-b8b3-e0833120dbe5 None admin] RBAC: Authorizing `identity:create_grant(project_id=463d7341c0f949bdb062115f14db583f, user_id=216d4237a0a44e7eaf170e6dcd700023, role_id=ef689f96442945e8a7bc8cbfee13d909)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:18.009930 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f694afeb-90b3-4582-b8b3-e0833120dbe5 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:18.011682 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19b323a-ceda-43eb-972a-7edbf73c03a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:18.012455 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b323a-ceda-43eb-972a-7edbf73c03a5 None admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:18.012830 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b323a-ceda-43eb-972a-7edbf73c03a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:18.013201 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b323a-ceda-43eb-972a-7edbf73c03a5 None admin] PATH_INFO: `/v3/projects/410027d6c764410ea5b76db0171903df/users/1fcb9022bfe743ae9405fc8470a5e016/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:18.015256 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19b323a-ceda-43eb-972a-7edbf73c03a5 None admin] RBAC: Authorizing `identity:create_grant(project_id=410027d6c764410ea5b76db0171903df, user_id=1fcb9022bfe743ae9405fc8470a5e016, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:18.016829 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19b323a-ceda-43eb-972a-7edbf73c03a5 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:18.086239 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 44/85] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Oct 12 00:01:17 2022] PUT /identity/v3/projects/463d7341c0f949bdb062115f14db583f/users/216d4237a0a44e7eaf170e6dcd700023/roles/ef689f96442945e8a7bc8cbfee13d909 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:01:18.116717 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 42/86] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Oct 12 00:01:17 2022] PUT /identity/v3/projects/410027d6c764410ea5b76db0171903df/users/1fcb9022bfe743ae9405fc8470a5e016/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:01:19.577209 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4cfaaa-d03d-4042-95a9-ae874a1dd039 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:19.577689 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4cfaaa-d03d-4042-95a9-ae874a1dd039 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:19.577792 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4cfaaa-d03d-4042-95a9-ae874a1dd039 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:19.579231 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 45/87] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:01:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:01:19.584623 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dadccf6-2508-4361-900b-cea926fb882d None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:19.585536 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dadccf6-2508-4361-900b-cea926fb882d None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:19.585713 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dadccf6-2508-4361-900b-cea926fb882d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:19.612607 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-6dadccf6-2508-4361-900b-cea926fb882d None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:01:19.613828 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-6dadccf6-2508-4361-900b-cea926fb882d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:01:19.703849 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 43/88] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:01:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 00:01:20.229852 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4cfaaa-d03d-4042-95a9-ae874a1dd039 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:20.250061 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:20.356182 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4cfaaa-d03d-4042-95a9-ae874a1dd039 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:20.402454 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26903eb-b7c3-4420-8742-e600b21cdf55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:20.403104 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26903eb-b7c3-4420-8742-e600b21cdf55 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:20.403390 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26903eb-b7c3-4420-8742-e600b21cdf55 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:20.403686 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26903eb-b7c3-4420-8742-e600b21cdf55 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:20.420430 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26903eb-b7c3-4420-8742-e600b21cdf55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 00:01:20.425340 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-b26903eb-b7c3-4420-8742-e600b21cdf55 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 12 00:01:20.425340 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:01:20.425340 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:01:20.425340 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:01:20.425340 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:01:20.425340 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:01:20.425340 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:01:20.425340 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:01:20.425340 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:01:20.425340 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:01:20.425340 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:01:20.425340 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:01:20.425340 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 00:01:20.425340 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 00:01:20.425340 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 00:01:20.425340 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 00:01:20.426162 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 00:01:20.426162 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 00:01:20.426162 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 00:01:20.426162 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 00:01:20.426162 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:01:20.426162 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:01:20.426162 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:01:20.426162 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:01:20.426162 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:01:20.426162 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:01:20.426162 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:01:20.426162 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:01:20.426162 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:01:20.426162 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:01:20.426162 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:01:20.426162 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:01:20.428324 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:01:20.428324 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:01:20.428324 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:01:20.428324 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:01:20.428324 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 00:01:20.428324 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 00:01:20.428324 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 00:01:20.428324 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 00:01:20.428324 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 00:01:20.428324 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 00:01:20.428324 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 12 00:01:20.428324 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:01:20.428324 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 46/89] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Oct 12 00:01:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:01:20.438826 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dadccf6-2508-4361-900b-cea926fb882d None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:20.592038 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:20.688387 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dadccf6-2508-4361-900b-cea926fb882d None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:20.734789 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138c7054-938e-4ace-b187-7cc20ca411a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:20.735436 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138c7054-938e-4ace-b187-7cc20ca411a5 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:20.735721 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138c7054-938e-4ace-b187-7cc20ca411a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:20.735993 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138c7054-938e-4ace-b187-7cc20ca411a5 None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:20.736662 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138c7054-938e-4ace-b187-7cc20ca411a5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:01:20.736965 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138c7054-938e-4ace-b187-7cc20ca411a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:20.738352 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138c7054-938e-4ace-b187-7cc20ca411a5 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:20.754617 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 44/90] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Oct 12 00:01:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:20.760925 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26903eb-b7c3-4420-8742-e600b21cdf55 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:20.830695 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4595f015-d43c-43e1-977c-71868af2cc58 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:20.830997 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4595f015-d43c-43e1-977c-71868af2cc58 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:20.831246 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4595f015-d43c-43e1-977c-71868af2cc58 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:20.832636 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 45/91] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:01:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:01:20.842037 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ba419e-3e9a-43d5-bb1f-6cdfdf8ca21a None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:20.842322 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ba419e-3e9a-43d5-bb1f-6cdfdf8ca21a None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:20.842605 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ba419e-3e9a-43d5-bb1f-6cdfdf8ca21a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:20.867271 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:20.873748 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-c6ba419e-3e9a-43d5-bb1f-6cdfdf8ca21a None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:01:20.874969 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-c6ba419e-3e9a-43d5-bb1f-6cdfdf8ca21a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:01:20.921321 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26903eb-b7c3-4420-8742-e600b21cdf55 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:20.969858 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0c3e50-165a-4195-a75f-af0945cd7c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:20.970858 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0c3e50-165a-4195-a75f-af0945cd7c20 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:20.971331 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0c3e50-165a-4195-a75f-af0945cd7c20 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:20.971842 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0c3e50-165a-4195-a75f-af0945cd7c20 None admin] PATH_INFO: `/v3/users/216d4237a0a44e7eaf170e6dcd700023` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:20.974960 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0c3e50-165a-4195-a75f-af0945cd7c20 None admin] RBAC: Authorizing `identity:get_user(user_id=216d4237a0a44e7eaf170e6dcd700023)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:20.977276 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0c3e50-165a-4195-a75f-af0945cd7c20 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:20.980575 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 47/92] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Oct 12 00:01:20 2022] GET /identity/v3/users/216d4237a0a44e7eaf170e6dcd700023 => generated 275 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:20.982724 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff4f41-caf4-4183-a836-aa295fe07d71 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:20.984979 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff4f41-caf4-4183-a836-aa295fe07d71 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:20.985546 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff4f41-caf4-4183-a836-aa295fe07d71 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:20.988160 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 48/93] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:01:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:01:20.989069 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ff4f41-caf4-4183-a836-aa295fe07d71 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:21.030815 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 46/94] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:01:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 12 00:01:21.032429 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74432e0b-25d7-4482-949f-3f624af94194 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:21.033750 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74432e0b-25d7-4482-949f-3f624af94194 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:21.036085 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74432e0b-25d7-4482-949f-3f624af94194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:21.073531 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-74432e0b-25d7-4482-949f-3f624af94194 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:01:21.075170 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-74432e0b-25d7-4482-949f-3f624af94194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:01:21.123796 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:21.167847 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 47/95] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:01:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 00:01:21.197015 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ff4f41-caf4-4183-a836-aa295fe07d71 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:21.255638 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5e6121-8eb9-47c0-82f9-f10b4b28f1a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:21.256274 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e6121-8eb9-47c0-82f9-f10b4b28f1a2 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:21.256557 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e6121-8eb9-47c0-82f9-f10b4b28f1a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:21.257358 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e6121-8eb9-47c0-82f9-f10b4b28f1a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:21.281839 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:21.359193 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5e6121-8eb9-47c0-82f9-f10b4b28f1a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:21.360725 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5e6121-8eb9-47c0-82f9-f10b4b28f1a2 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:21.468527 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74432e0b-25d7-4482-949f-3f624af94194 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:21.514526 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 49/96] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Oct 12 00:01:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 523 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 12 00:01:21.514526 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69ec168-ba15-4605-953a-29bf34b98440 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:21.514526 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69ec168-ba15-4605-953a-29bf34b98440 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:21.514526 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69ec168-ba15-4605-953a-29bf34b98440 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:21.515289 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 50/97] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:01:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:01:21.520221 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69ec168-ba15-4605-953a-29bf34b98440 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:21.575272 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:21.616783 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74432e0b-25d7-4482-949f-3f624af94194 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:21.642834 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69ec168-ba15-4605-953a-29bf34b98440 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:21.652060 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e556685-14fa-4b9c-a592-e6fd72279cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:21.652699 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e556685-14fa-4b9c-a592-e6fd72279cf3 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:21.652982 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e556685-14fa-4b9c-a592-e6fd72279cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:21.653258 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e556685-14fa-4b9c-a592-e6fd72279cf3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:21.663700 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e556685-14fa-4b9c-a592-e6fd72279cf3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 00:01:21.669035 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-5e556685-14fa-4b9c-a592-e6fd72279cf3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 12 00:01:21.669035 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:01:21.669035 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:01:21.669035 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:01:21.669035 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:01:21.669035 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:01:21.669035 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:01:21.669035 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:01:21.669035 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:01:21.669035 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:01:21.669035 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:01:21.669035 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:01:21.669035 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 00:01:21.669035 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 00:01:21.669035 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 00:01:21.669035 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 00:01:21.669883 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 00:01:21.669883 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 00:01:21.669883 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 00:01:21.669883 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 00:01:21.669883 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:01:21.669883 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:01:21.669883 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:01:21.669883 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:01:21.669883 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:01:21.669883 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:01:21.669883 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:01:21.669883 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:01:21.669883 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:01:21.669883 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:01:21.669883 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:01:21.669883 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:01:21.671045 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:01:21.671045 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:01:21.671045 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:01:21.671045 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:01:21.671045 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 00:01:21.671045 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 00:01:21.671045 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 00:01:21.671045 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 00:01:21.671045 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 00:01:21.671045 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 00:01:21.671045 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 12 00:01:21.671045 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:01:21.671045 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 48/98] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Oct 12 00:01:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 00:01:21.672560 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e328c3c3-0ed9-4f15-86bf-ca009a461ea5 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:21.673361 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e328c3c3-0ed9-4f15-86bf-ca009a461ea5 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:21.673646 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e328c3c3-0ed9-4f15-86bf-ca009a461ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:21.684659 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ff599b-d2ad-4ee7-9e0b-4ee20e0a3aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:21.685296 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ff599b-d2ad-4ee7-9e0b-4ee20e0a3aa3 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:21.685578 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ff599b-d2ad-4ee7-9e0b-4ee20e0a3aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:21.685850 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ff599b-d2ad-4ee7-9e0b-4ee20e0a3aa3 None admin] PATH_INFO: `/v3/projects/410027d6c764410ea5b76db0171903df` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:21.686966 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ff599b-d2ad-4ee7-9e0b-4ee20e0a3aa3 None admin] RBAC: Authorizing `identity:get_project(project_id=410027d6c764410ea5b76db0171903df)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:21.688442 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ff599b-d2ad-4ee7-9e0b-4ee20e0a3aa3 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:21.690070 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 51/99] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Oct 12 00:01:21 2022] GET /identity/v3/projects/410027d6c764410ea5b76db0171903df => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:21.690826 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ff599b-d2ad-4ee7-9e0b-4ee20e0a3aa3 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:21.697474 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-e328c3c3-0ed9-4f15-86bf-ca009a461ea5 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:01:21.699015 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-e328c3c3-0ed9-4f15-86bf-ca009a461ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:01:21.717690 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:21.779334 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 49/100] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:01:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 00:01:21.780590 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b882f01-5442-47c6-9f85-f14a5a477e9a None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:21.780875 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b882f01-5442-47c6-9f85-f14a5a477e9a None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:21.781146 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b882f01-5442-47c6-9f85-f14a5a477e9a None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:21.782404 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 50/101] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:01:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:01:21.786581 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b882f01-5442-47c6-9f85-f14a5a477e9a None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:21.811868 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ff599b-d2ad-4ee7-9e0b-4ee20e0a3aa3 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:21.831930 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:21.855421 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9529e2a7-296c-41e6-ac68-eb705d01bb4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:21.856518 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9529e2a7-296c-41e6-ac68-eb705d01bb4a None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:21.856995 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9529e2a7-296c-41e6-ac68-eb705d01bb4a None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:21.857459 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9529e2a7-296c-41e6-ac68-eb705d01bb4a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:21.870068 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9529e2a7-296c-41e6-ac68-eb705d01bb4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 00:01:21.876300 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-9529e2a7-296c-41e6-ac68-eb705d01bb4a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 12 00:01:21.876300 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:01:21.876300 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:01:21.876300 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:01:21.876300 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:01:21.876300 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:01:21.876300 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:01:21.876300 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:01:21.876300 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:01:21.876300 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:01:21.876300 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:01:21.876300 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:01:21.876300 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 00:01:21.876300 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 00:01:21.876300 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 00:01:21.876300 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 00:01:21.877143 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 00:01:21.877143 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 00:01:21.877143 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 00:01:21.877143 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 00:01:21.877143 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:01:21.877143 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:01:21.877143 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:01:21.877143 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:01:21.877143 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:01:21.877143 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:01:21.877143 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:01:21.877143 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:01:21.877143 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:01:21.877143 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:01:21.877143 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:01:21.877143 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:01:21.877900 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:01:21.877900 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:01:21.877900 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:01:21.877900 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:01:21.877900 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 00:01:21.877900 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 00:01:21.877900 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 00:01:21.877900 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 00:01:21.877900 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 00:01:21.877900 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 00:01:21.877900 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 12 00:01:21.877900 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:01:21.880813 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 52/102] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Oct 12 00:01:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:01:21.882626 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9529e2a7-296c-41e6-ac68-eb705d01bb4a None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:21.921335 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b882f01-5442-47c6-9f85-f14a5a477e9a None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:21.957503 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e593e24c-af9b-49a6-a2d1-6c351436f0e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:21.958119 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e593e24c-af9b-49a6-a2d1-6c351436f0e7 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:21.958401 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e593e24c-af9b-49a6-a2d1-6c351436f0e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:21.958672 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e593e24c-af9b-49a6-a2d1-6c351436f0e7 None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:21.959341 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e593e24c-af9b-49a6-a2d1-6c351436f0e7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:01:21.959673 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e593e24c-af9b-49a6-a2d1-6c351436f0e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:21.961084 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e593e24c-af9b-49a6-a2d1-6c351436f0e7 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:21.969133 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 51/103] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Oct 12 00:01:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:21.972610 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddae97a-c30c-4181-8ed7-d7d2d1f8d1f8 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:21.972894 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddae97a-c30c-4181-8ed7-d7d2d1f8d1f8 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:21.973165 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddae97a-c30c-4181-8ed7-d7d2d1f8d1f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:21.992140 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-9ddae97a-c30c-4181-8ed7-d7d2d1f8d1f8 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:01:21.992985 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9529e2a7-296c-41e6-ac68-eb705d01bb4a None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:21.993345 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-9ddae97a-c30c-4181-8ed7-d7d2d1f8d1f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:01:22.003672 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:22.028284 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa519efc-9cd9-4d12-a576-8d72d9fe74da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:22.028893 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa519efc-9cd9-4d12-a576-8d72d9fe74da None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:22.029172 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa519efc-9cd9-4d12-a576-8d72d9fe74da None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:22.029506 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa519efc-9cd9-4d12-a576-8d72d9fe74da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:22.030175 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa519efc-9cd9-4d12-a576-8d72d9fe74da None admin] RBAC: Adding query filter params (role.id=ef689f96442945e8a7bc8cbfee13d909, scope.project.id=410027d6c764410ea5b76db0171903df, user.id=216d4237a0a44e7eaf170e6dcd700023) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:01:22.030483 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa519efc-9cd9-4d12-a576-8d72d9fe74da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:22.032095 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa519efc-9cd9-4d12-a576-8d72d9fe74da None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:22.043404 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 53/104] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Oct 12 00:01:21 2022] GET /identity/v3/role_assignments?user.id=216d4237a0a44e7eaf170e6dcd700023&scope.project.id=410027d6c764410ea5b76db0171903df&role.id=ef689f96442945e8a7bc8cbfee13d909 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:22.043404 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa519efc-9cd9-4d12-a576-8d72d9fe74da None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:22.078603 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 52/105] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:01:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 00:01:22.079958 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ca6d41-6ebc-42a0-9abd-d1cfe124fd1e None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:22.080247 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ca6d41-6ebc-42a0-9abd-d1cfe124fd1e None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:22.080523 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ca6d41-6ebc-42a0-9abd-d1cfe124fd1e None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:22.081853 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 53/106] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:01:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:01:22.082571 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ca6d41-6ebc-42a0-9abd-d1cfe124fd1e None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:22.098321 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:22.181348 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa519efc-9cd9-4d12-a576-8d72d9fe74da None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:22.213475 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ca6d41-6ebc-42a0-9abd-d1cfe124fd1e None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:22.238592 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd405306-2ee5-4dc3-a1e0-fc109890ad8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:22.239220 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd405306-2ee5-4dc3-a1e0-fc109890ad8a None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:22.239535 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd405306-2ee5-4dc3-a1e0-fc109890ad8a None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:22.239815 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd405306-2ee5-4dc3-a1e0-fc109890ad8a None admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:22.240511 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd405306-2ee5-4dc3-a1e0-fc109890ad8a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:01:22.240787 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd405306-2ee5-4dc3-a1e0-fc109890ad8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:22.242762 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd405306-2ee5-4dc3-a1e0-fc109890ad8a None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:22.257903 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 54/107] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Oct 12 00:01:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:22.259177 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd405306-2ee5-4dc3-a1e0-fc109890ad8a None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:22.272501 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7c6d44-b0a5-4e2e-90f9-95a4ca8f438b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:22.273127 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7c6d44-b0a5-4e2e-90f9-95a4ca8f438b None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:22.273412 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7c6d44-b0a5-4e2e-90f9-95a4ca8f438b None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:22.273693 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7c6d44-b0a5-4e2e-90f9-95a4ca8f438b None admin] PATH_INFO: `/v3/users/216d4237a0a44e7eaf170e6dcd700023` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:22.275230 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7c6d44-b0a5-4e2e-90f9-95a4ca8f438b None admin] RBAC: Authorizing `identity:get_user(user_id=216d4237a0a44e7eaf170e6dcd700023)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:22.277326 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7c6d44-b0a5-4e2e-90f9-95a4ca8f438b None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:22.279825 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 54/108] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Oct 12 00:01:22 2022] GET /identity/v3/users/216d4237a0a44e7eaf170e6dcd700023 => generated 275 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:22.280927 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab6f003-5efb-44ea-a750-f751da750238 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:22.281212 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab6f003-5efb-44ea-a750-f751da750238 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:22.281490 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab6f003-5efb-44ea-a750-f751da750238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:22.302119 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-aab6f003-5efb-44ea-a750-f751da750238 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:01:22.303298 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-aab6f003-5efb-44ea-a750-f751da750238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:01:22.311579 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:22.370517 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:22.372408 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 55/109] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:01:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 00:01:22.372914 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab6f003-5efb-44ea-a750-f751da750238 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:22.379151 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd405306-2ee5-4dc3-a1e0-fc109890ad8a None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:22.417921 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0efd6a4-3d32-4c37-9cbc-5263912415d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:22.418562 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0efd6a4-3d32-4c37-9cbc-5263912415d5 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:22.418845 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0efd6a4-3d32-4c37-9cbc-5263912415d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:22.419121 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0efd6a4-3d32-4c37-9cbc-5263912415d5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:22.420261 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0efd6a4-3d32-4c37-9cbc-5263912415d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:22.421708 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0efd6a4-3d32-4c37-9cbc-5263912415d5 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:22.423284 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 55/110] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:01:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:22.424073 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0efd6a4-3d32-4c37-9cbc-5263912415d5 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:22.481218 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab6f003-5efb-44ea-a750-f751da750238 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:22.516358 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be3e07b-5bec-42e6-80d1-6d74c8b57def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:22.517589 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be3e07b-5bec-42e6-80d1-6d74c8b57def None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:22.518038 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be3e07b-5bec-42e6-80d1-6d74c8b57def None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:22.518323 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be3e07b-5bec-42e6-80d1-6d74c8b57def None admin] PATH_INFO: `/v3/users/06b6677a643746bbbbe5b869a79b1da6` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:22.529341 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0efd6a4-3d32-4c37-9cbc-5263912415d5 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:22.530050 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:22.532419 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be3e07b-5bec-42e6-80d1-6d74c8b57def None admin] RBAC: Authorizing `identity:get_user(user_id=06b6677a643746bbbbe5b869a79b1da6)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:22.534042 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be3e07b-5bec-42e6-80d1-6d74c8b57def None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:22.535721 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 56/111] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Oct 12 00:01:22 2022] GET /identity/v3/users/06b6677a643746bbbbe5b869a79b1da6 => generated 289 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:22.536555 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be3e07b-5bec-42e6-80d1-6d74c8b57def None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:22.570838 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d865b2-ad38-4fce-beb6-32e2cc21fcbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:22.571478 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d865b2-ad38-4fce-beb6-32e2cc21fcbd None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:22.571838 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d865b2-ad38-4fce-beb6-32e2cc21fcbd None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:22.572132 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d865b2-ad38-4fce-beb6-32e2cc21fcbd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:22.606250 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:22.644492 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d865b2-ad38-4fce-beb6-32e2cc21fcbd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:22.646003 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d865b2-ad38-4fce-beb6-32e2cc21fcbd None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:22.649116 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be3e07b-5bec-42e6-80d1-6d74c8b57def None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:22.682160 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b526115b-89a4-4764-832e-d4e290deb198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:22.682785 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b526115b-89a4-4764-832e-d4e290deb198 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:22.683061 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b526115b-89a4-4764-832e-d4e290deb198 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:22.683337 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b526115b-89a4-4764-832e-d4e290deb198 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:22.693097 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b526115b-89a4-4764-832e-d4e290deb198 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 00:01:22.698521 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-b526115b-89a4-4764-832e-d4e290deb198 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 12 00:01:22.698521 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:01:22.698521 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:01:22.698521 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:01:22.698521 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:01:22.698521 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:01:22.698521 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:01:22.698521 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:01:22.698521 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:01:22.698521 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:01:22.698521 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:01:22.698521 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:01:22.698521 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 00:01:22.698521 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 00:01:22.698521 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 00:01:22.698521 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 00:01:22.699474 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 00:01:22.699474 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 00:01:22.699474 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 00:01:22.699474 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 00:01:22.699474 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:01:22.699474 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:01:22.699474 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:01:22.699474 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:01:22.699474 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:01:22.699474 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:01:22.699474 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:01:22.699474 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:01:22.699474 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:01:22.699474 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:01:22.699474 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:01:22.699474 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:01:22.700229 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:01:22.700229 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:01:22.700229 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:01:22.700229 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:01:22.700229 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 00:01:22.700229 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 00:01:22.700229 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 00:01:22.700229 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 00:01:22.700229 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 00:01:22.700229 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 00:01:22.700229 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 12 00:01:22.700229 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:01:22.700229 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 57/112] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Oct 12 00:01:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:01:22.700846 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b526115b-89a4-4764-832e-d4e290deb198 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:22.745830 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 56/113] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Oct 12 00:01:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 12 00:01:22.746932 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d865b2-ad38-4fce-beb6-32e2cc21fcbd None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:22.761843 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:22.801767 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b526115b-89a4-4764-832e-d4e290deb198 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:22.809898 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:22.833882 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df889294-141c-42c6-80cb-c771dd17ae1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:22.834479 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df889294-141c-42c6-80cb-c771dd17ae1f None admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:22.834784 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df889294-141c-42c6-80cb-c771dd17ae1f None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:22.835053 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df889294-141c-42c6-80cb-c771dd17ae1f None admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:22.835728 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df889294-141c-42c6-80cb-c771dd17ae1f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:22.837200 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df889294-141c-42c6-80cb-c771dd17ae1f None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:22.842457 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d865b2-ad38-4fce-beb6-32e2cc21fcbd None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:22.876510 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a92e4c1-8a3a-4a8b-9807-e20aad28b4c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:22.877143 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a92e4c1-8a3a-4a8b-9807-e20aad28b4c9 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:22.877428 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a92e4c1-8a3a-4a8b-9807-e20aad28b4c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:22.877706 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a92e4c1-8a3a-4a8b-9807-e20aad28b4c9 None admin] PATH_INFO: `/v3/projects/410027d6c764410ea5b76db0171903df` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:22.878882 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a92e4c1-8a3a-4a8b-9807-e20aad28b4c9 None admin] RBAC: Authorizing `identity:get_project(project_id=410027d6c764410ea5b76db0171903df)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:22.880484 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a92e4c1-8a3a-4a8b-9807-e20aad28b4c9 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:22.882127 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 57/114] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Oct 12 00:01:22 2022] GET /identity/v3/projects/410027d6c764410ea5b76db0171903df => generated 298 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:22.882889 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a92e4c1-8a3a-4a8b-9807-e20aad28b4c9 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:22.921554 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 58/115] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Oct 12 00:01:22 2022] POST /identity/v3/projects => generated 302 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:01:22.922384 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df889294-141c-42c6-80cb-c771dd17ae1f None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:23.008253 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a92e4c1-8a3a-4a8b-9807-e20aad28b4c9 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:23.036964 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df889294-141c-42c6-80cb-c771dd17ae1f None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:23.046674 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2221776a-68fa-4a9a-a58b-a63f298fb90f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:23.047397 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2221776a-68fa-4a9a-a58b-a63f298fb90f None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:23.047686 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2221776a-68fa-4a9a-a58b-a63f298fb90f None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:23.047960 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2221776a-68fa-4a9a-a58b-a63f298fb90f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:23.051064 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:23.063499 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:23.063677 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2221776a-68fa-4a9a-a58b-a63f298fb90f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 00:01:23.066212 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-2221776a-68fa-4a9a-a58b-a63f298fb90f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 00:01:23.066212 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:01:23.066212 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:01:23.066212 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:01:23.066212 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:01:23.066212 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:01:23.066212 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:01:23.066212 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:01:23.066212 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:01:23.066212 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:01:23.066212 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:01:23.066212 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:01:23.066212 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 00:01:23.066212 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 00:01:23.066212 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 00:01:23.066212 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 00:01:23.067072 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 00:01:23.067072 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 00:01:23.067072 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 00:01:23.067072 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 00:01:23.067072 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:01:23.067072 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:01:23.067072 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:01:23.067072 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:01:23.067072 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:01:23.067072 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:01:23.067072 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:01:23.067072 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:01:23.067072 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:01:23.067072 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:01:23.067072 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:01:23.067072 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:01:23.068306 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:01:23.068306 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:01:23.068306 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:01:23.068306 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:01:23.068306 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 00:01:23.068306 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 00:01:23.068306 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 00:01:23.068306 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 00:01:23.068306 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 00:01:23.068306 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 00:01:23.068306 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 00:01:23.068306 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:01:23.068306 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 58/116] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Oct 12 00:01:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:01:23.072542 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2221776a-68fa-4a9a-a58b-a63f298fb90f None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:23.083621 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53b455d-47f8-42e7-9b2a-f1bb05591825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:23.084363 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53b455d-47f8-42e7-9b2a-f1bb05591825 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:23.084535 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53b455d-47f8-42e7-9b2a-f1bb05591825 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:23.085345 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53b455d-47f8-42e7-9b2a-f1bb05591825 None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:23.086204 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53b455d-47f8-42e7-9b2a-f1bb05591825 None admin] RBAC: Adding query filter params (name=member) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:01:23.086412 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53b455d-47f8-42e7-9b2a-f1bb05591825 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:23.087725 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53b455d-47f8-42e7-9b2a-f1bb05591825 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:23.096644 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 59/117] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Oct 12 00:01:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:23.102742 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53b455d-47f8-42e7-9b2a-f1bb05591825 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:23.191956 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2221776a-68fa-4a9a-a58b-a63f298fb90f None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:23.208656 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53b455d-47f8-42e7-9b2a-f1bb05591825 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:23.229622 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2b1cb0-c0a5-48ef-8bcd-02d8fbaafd65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:23.230261 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b1cb0-c0a5-48ef-8bcd-02d8fbaafd65 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:23.230543 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b1cb0-c0a5-48ef-8bcd-02d8fbaafd65 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:23.230819 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b1cb0-c0a5-48ef-8bcd-02d8fbaafd65 None admin] PATH_INFO: `/v3/projects/410027d6c764410ea5b76db0171903df` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:23.232116 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2b1cb0-c0a5-48ef-8bcd-02d8fbaafd65 None admin] RBAC: Authorizing `identity:get_project(project_id=410027d6c764410ea5b76db0171903df)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:23.233722 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2b1cb0-c0a5-48ef-8bcd-02d8fbaafd65 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:23.235796 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 59/118] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Oct 12 00:01:23 2022] GET /identity/v3/projects/410027d6c764410ea5b76db0171903df => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:23.241455 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2b1cb0-c0a5-48ef-8bcd-02d8fbaafd65 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:23.242714 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f73550-e2a1-449f-a38f-55a9e1baefe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:23.243403 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f73550-e2a1-449f-a38f-55a9e1baefe8 None admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:23.243687 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f73550-e2a1-449f-a38f-55a9e1baefe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:23.243957 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f73550-e2a1-449f-a38f-55a9e1baefe8 None admin] PATH_INFO: `/v3/projects/410027d6c764410ea5b76db0171903df/users/06b6677a643746bbbbe5b869a79b1da6/roles/217d497bae80496a9d3772fbb90678e1` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:23.245953 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f73550-e2a1-449f-a38f-55a9e1baefe8 None admin] RBAC: Authorizing `identity:create_grant(project_id=410027d6c764410ea5b76db0171903df, user_id=06b6677a643746bbbbe5b869a79b1da6, role_id=217d497bae80496a9d3772fbb90678e1)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:23.247439 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f73550-e2a1-449f-a38f-55a9e1baefe8 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:23.304374 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:23.339392 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 60/119] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Oct 12 00:01:23 2022] PUT /identity/v3/projects/410027d6c764410ea5b76db0171903df/users/06b6677a643746bbbbe5b869a79b1da6/roles/217d497bae80496a9d3772fbb90678e1 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:01:23.339392 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f73550-e2a1-449f-a38f-55a9e1baefe8 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:23.358424 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2b1cb0-c0a5-48ef-8bcd-02d8fbaafd65 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:23.374819 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:23.408360 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16041646-018f-415f-ab28-3245463a750b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:23.409002 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16041646-018f-415f-ab28-3245463a750b None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:23.409287 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16041646-018f-415f-ab28-3245463a750b None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:23.409565 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16041646-018f-415f-ab28-3245463a750b None admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:23.410247 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16041646-018f-415f-ab28-3245463a750b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:01:23.410553 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16041646-018f-415f-ab28-3245463a750b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:23.412189 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16041646-018f-415f-ab28-3245463a750b None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:23.424291 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 60/120] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Oct 12 00:01:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:23.425628 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16041646-018f-415f-ab28-3245463a750b None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:23.470060 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f73550-e2a1-449f-a38f-55a9e1baefe8 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:23.524735 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4539af-5c10-4f1d-90a3-2e22aecbedd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:23.525975 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4539af-5c10-4f1d-90a3-2e22aecbedd6 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:23.526265 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4539af-5c10-4f1d-90a3-2e22aecbedd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:23.526587 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4539af-5c10-4f1d-90a3-2e22aecbedd6 None admin] PATH_INFO: `/v3/users/216d4237a0a44e7eaf170e6dcd700023` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:23.528246 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4539af-5c10-4f1d-90a3-2e22aecbedd6 None admin] RBAC: Authorizing `identity:get_user(user_id=216d4237a0a44e7eaf170e6dcd700023)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:23.530459 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4539af-5c10-4f1d-90a3-2e22aecbedd6 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:23.532270 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 61/121] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Oct 12 00:01:23 2022] GET /identity/v3/users/216d4237a0a44e7eaf170e6dcd700023 => generated 275 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:23.533427 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4539af-5c10-4f1d-90a3-2e22aecbedd6 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:23.555143 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16041646-018f-415f-ab28-3245463a750b None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:23.582727 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:23.593383 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d50d48-dcde-4736-b1e1-dad3acac1eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:23.593933 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d50d48-dcde-4736-b1e1-dad3acac1eda None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:23.594217 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d50d48-dcde-4736-b1e1-dad3acac1eda None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:23.594496 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d50d48-dcde-4736-b1e1-dad3acac1eda None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:23.595228 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d50d48-dcde-4736-b1e1-dad3acac1eda None admin] RBAC: Adding query filter params (role.id=81991615ac0b46818426bf37015df930, scope.project.id=410027d6c764410ea5b76db0171903df, user.id=216d4237a0a44e7eaf170e6dcd700023) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:01:23.595638 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d50d48-dcde-4736-b1e1-dad3acac1eda None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:23.597224 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d50d48-dcde-4736-b1e1-dad3acac1eda None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:23.603669 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 61/122] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Oct 12 00:01:23 2022] GET /identity/v3/role_assignments?user.id=216d4237a0a44e7eaf170e6dcd700023&scope.project.id=410027d6c764410ea5b76db0171903df&role.id=81991615ac0b46818426bf37015df930 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:23.604500 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d50d48-dcde-4736-b1e1-dad3acac1eda None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:23.668902 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4539af-5c10-4f1d-90a3-2e22aecbedd6 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:23.672331 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:23.714598 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaef512-49b3-4584-be1d-8599bf6d837b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:23.715250 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaef512-49b3-4584-be1d-8599bf6d837b None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:23.715486 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d50d48-dcde-4736-b1e1-dad3acac1eda None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:23.715590 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaef512-49b3-4584-be1d-8599bf6d837b None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:23.715814 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaef512-49b3-4584-be1d-8599bf6d837b None admin] PATH_INFO: `/v3/users/1fcb9022bfe743ae9405fc8470a5e016` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:23.717220 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaef512-49b3-4584-be1d-8599bf6d837b None admin] RBAC: Authorizing `identity:get_user(user_id=1fcb9022bfe743ae9405fc8470a5e016)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:23.719720 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaef512-49b3-4584-be1d-8599bf6d837b None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:23.721417 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 62/123] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Oct 12 00:01:23 2022] GET /identity/v3/users/1fcb9022bfe743ae9405fc8470a5e016 => generated 247 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:23.727089 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaef512-49b3-4584-be1d-8599bf6d837b None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:23.810704 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a8b362-9229-438e-a86d-f120af0d4460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:23.811303 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a8b362-9229-438e-a86d-f120af0d4460 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:23.811617 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a8b362-9229-438e-a86d-f120af0d4460 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:23.811895 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a8b362-9229-438e-a86d-f120af0d4460 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:23.856395 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaef512-49b3-4584-be1d-8599bf6d837b None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:23.897357 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd075f6-1853-4beb-8534-146b7e5e395c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:23.898409 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd075f6-1853-4beb-8534-146b7e5e395c None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:23.898884 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd075f6-1853-4beb-8534-146b7e5e395c None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:23.899345 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd075f6-1853-4beb-8534-146b7e5e395c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:23.900940 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:23.913558 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a8b362-9229-438e-a86d-f120af0d4460 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:23.915035 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a8b362-9229-438e-a86d-f120af0d4460 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:23.942420 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:23.975158 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd075f6-1853-4beb-8534-146b7e5e395c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:23.976997 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd075f6-1853-4beb-8534-146b7e5e395c None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:24.027150 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 62/124] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Oct 12 00:01:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 12 00:01:24.033547 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a8b362-9229-438e-a86d-f120af0d4460 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:24.095547 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 63/125] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Oct 12 00:01:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 12 00:01:24.104138 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd075f6-1853-4beb-8534-146b7e5e395c None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:24.153413 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:24.168471 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a8b362-9229-438e-a86d-f120af0d4460 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:24.190359 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:24.210695 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b684df-60b1-45cb-ab3c-edb64cdd90eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:24.211644 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b684df-60b1-45cb-ab3c-edb64cdd90eb None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:24.212095 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b684df-60b1-45cb-ab3c-edb64cdd90eb None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:24.212536 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b684df-60b1-45cb-ab3c-edb64cdd90eb None admin] PATH_INFO: `/v3/projects/463d7341c0f949bdb062115f14db583f` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:24.214050 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b684df-60b1-45cb-ab3c-edb64cdd90eb None admin] RBAC: Authorizing `identity:get_project(project_id=463d7341c0f949bdb062115f14db583f)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:24.216371 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b684df-60b1-45cb-ab3c-edb64cdd90eb None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:24.218198 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 63/126] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Oct 12 00:01:24 2022] GET /identity/v3/projects/463d7341c0f949bdb062115f14db583f => generated 312 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:24.224514 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b684df-60b1-45cb-ab3c-edb64cdd90eb None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:24.227274 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd075f6-1853-4beb-8534-146b7e5e395c None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:24.274081 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdd593c-c646-4b29-96cf-12c842e39efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:24.274822 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdd593c-c646-4b29-96cf-12c842e39efd None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:24.275192 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdd593c-c646-4b29-96cf-12c842e39efd None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:24.275562 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdd593c-c646-4b29-96cf-12c842e39efd None admin] PATH_INFO: `/v3/projects/410027d6c764410ea5b76db0171903df` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:24.276804 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdd593c-c646-4b29-96cf-12c842e39efd None admin] RBAC: Authorizing `identity:get_project(project_id=410027d6c764410ea5b76db0171903df)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:24.278331 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdd593c-c646-4b29-96cf-12c842e39efd None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:24.285294 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 64/127] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Oct 12 00:01:24 2022] GET /identity/v3/projects/410027d6c764410ea5b76db0171903df => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:24.286566 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdd593c-c646-4b29-96cf-12c842e39efd None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:24.359317 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b684df-60b1-45cb-ab3c-edb64cdd90eb None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:24.392322 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdd593c-c646-4b29-96cf-12c842e39efd None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:24.405497 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fe450c-89c8-4437-a56a-55d62b05d782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:24.406343 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fe450c-89c8-4437-a56a-55d62b05d782 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:24.406950 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fe450c-89c8-4437-a56a-55d62b05d782 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:24.407424 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fe450c-89c8-4437-a56a-55d62b05d782 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:24.408684 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fe450c-89c8-4437-a56a-55d62b05d782 None admin] RBAC: Adding query filter params (role.id=ef689f96442945e8a7bc8cbfee13d909, scope.project.id=463d7341c0f949bdb062115f14db583f, user.id=216d4237a0a44e7eaf170e6dcd700023) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:01:24.409420 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fe450c-89c8-4437-a56a-55d62b05d782 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:24.411459 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fe450c-89c8-4437-a56a-55d62b05d782 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:24.419233 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 64/128] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Oct 12 00:01:24 2022] GET /identity/v3/role_assignments?user.id=216d4237a0a44e7eaf170e6dcd700023&scope.project.id=463d7341c0f949bdb062115f14db583f&role.id=ef689f96442945e8a7bc8cbfee13d909 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 00:01:24.434343 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a64c69-e922-4a4f-9ec2-cc84cea2e5fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:24.435443 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a64c69-e922-4a4f-9ec2-cc84cea2e5fc None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:24.435956 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a64c69-e922-4a4f-9ec2-cc84cea2e5fc None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:24.436433 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a64c69-e922-4a4f-9ec2-cc84cea2e5fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:24.437433 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a64c69-e922-4a4f-9ec2-cc84cea2e5fc None admin] RBAC: Adding query filter params (role.id=b2e41f0d7a304dc7af8b6a232049b6a2, scope.project.id=410027d6c764410ea5b76db0171903df, user.id=1fcb9022bfe743ae9405fc8470a5e016) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:01:24.437964 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a64c69-e922-4a4f-9ec2-cc84cea2e5fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:24.439770 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a64c69-e922-4a4f-9ec2-cc84cea2e5fc None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:24.449112 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 65/129] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Oct 12 00:01:24 2022] GET /identity/v3/role_assignments?user.id=1fcb9022bfe743ae9405fc8470a5e016&scope.project.id=410027d6c764410ea5b76db0171903df&role.id=b2e41f0d7a304dc7af8b6a232049b6a2 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:25.147779 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ddcdab-6132-44c8-84ec-9fa550b18df5 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:25.148303 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ddcdab-6132-44c8-84ec-9fa550b18df5 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:25.148386 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ddcdab-6132-44c8-84ec-9fa550b18df5 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:25.149685 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 65/130] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:01:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:01:25.154847 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1d61ad-f6db-42f5-a3e8-500daa628893 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:25.155284 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1d61ad-f6db-42f5-a3e8-500daa628893 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:25.155544 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1d61ad-f6db-42f5-a3e8-500daa628893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:25.161889 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:25.174424 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-7b1d61ad-f6db-42f5-a3e8-500daa628893 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:01:25.177429 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-7b1d61ad-f6db-42f5-a3e8-500daa628893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:01:25.240164 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 66/131] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:01:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 00:01:25.504674 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ddcdab-6132-44c8-84ec-9fa550b18df5 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:25.526828 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:25.606329 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ddcdab-6132-44c8-84ec-9fa550b18df5 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:25.643445 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af03db14-36e5-42f2-b5d7-d7f7ff91ba1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:25.644096 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af03db14-36e5-42f2-b5d7-d7f7ff91ba1e None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:25.644378 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af03db14-36e5-42f2-b5d7-d7f7ff91ba1e None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:25.644651 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af03db14-36e5-42f2-b5d7-d7f7ff91ba1e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:25.645754 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af03db14-36e5-42f2-b5d7-d7f7ff91ba1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:25.647190 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af03db14-36e5-42f2-b5d7-d7f7ff91ba1e None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:25.648799 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 66/132] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:01:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:25.654204 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1d61ad-f6db-42f5-a3e8-500daa628893 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:25.722577 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9918e3da-956d-4ab2-984a-045926660034 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:25.722858 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9918e3da-956d-4ab2-984a-045926660034 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:25.723143 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9918e3da-956d-4ab2-984a-045926660034 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:25.724521 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 67/133] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:01:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 00:01:25.729871 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ddecf7-c942-477c-a903-7886cfbc2f50 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:25.730157 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ddecf7-c942-477c-a903-7886cfbc2f50 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:25.730438 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ddecf7-c942-477c-a903-7886cfbc2f50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:25.748633 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-c6ddecf7-c942-477c-a903-7886cfbc2f50 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:01:25.749810 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1d61ad-f6db-42f5-a3e8-500daa628893 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:25.749915 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-c6ddecf7-c942-477c-a903-7886cfbc2f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:01:25.778323 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:25.782847 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66523e6-cb17-42dc-95ec-7eb9d95d3e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:25.783568 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66523e6-cb17-42dc-95ec-7eb9d95d3e4d None admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:25.783851 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66523e6-cb17-42dc-95ec-7eb9d95d3e4d None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:25.784124 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66523e6-cb17-42dc-95ec-7eb9d95d3e4d None admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:25.784781 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66523e6-cb17-42dc-95ec-7eb9d95d3e4d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:25.786235 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66523e6-cb17-42dc-95ec-7eb9d95d3e4d None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:25.816092 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 68/134] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:01:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 12 00:01:25.891589 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 67/135] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Oct 12 00:01:25 2022] POST /identity/v3/users => generated 283 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:01:26.089745 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ddecf7-c942-477c-a903-7886cfbc2f50 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:26.108639 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:26.190329 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ddecf7-c942-477c-a903-7886cfbc2f50 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:26.225914 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56d8d12-2eaa-4c6a-b203-c8e0da03a0df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:26.226567 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d8d12-2eaa-4c6a-b203-c8e0da03a0df None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:26.226853 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d8d12-2eaa-4c6a-b203-c8e0da03a0df None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:26.227130 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d8d12-2eaa-4c6a-b203-c8e0da03a0df None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:26.238136 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56d8d12-2eaa-4c6a-b203-c8e0da03a0df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 00:01:26.243113 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-c56d8d12-2eaa-4c6a-b203-c8e0da03a0df None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 12 00:01:26.243113 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:01:26.243113 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:01:26.243113 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:01:26.243113 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:01:26.243113 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:01:26.243113 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:01:26.243113 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:01:26.243113 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:01:26.243113 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:01:26.243113 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:01:26.243113 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:01:26.243113 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 00:01:26.243113 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 00:01:26.243113 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 00:01:26.243113 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 00:01:26.245994 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 00:01:26.245994 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 00:01:26.245994 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 00:01:26.245994 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 00:01:26.245994 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:01:26.245994 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:01:26.245994 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:01:26.245994 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:01:26.245994 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:01:26.245994 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:01:26.245994 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:01:26.245994 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:01:26.245994 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:01:26.245994 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:01:26.245994 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:01:26.245994 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:01:26.246772 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:01:26.246772 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:01:26.246772 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:01:26.246772 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:01:26.246772 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 00:01:26.246772 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 00:01:26.246772 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 00:01:26.246772 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 00:01:26.246772 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 00:01:26.246772 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 00:01:26.246772 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 12 00:01:26.246772 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:01:26.246772 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 69/136] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Oct 12 00:01:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:01:26.250371 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66523e6-cb17-42dc-95ec-7eb9d95d3e4d None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:26.359758 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66523e6-cb17-42dc-95ec-7eb9d95d3e4d None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:26.394986 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f3250e-0714-4bdf-919c-9796644ff7f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:26.395645 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f3250e-0714-4bdf-919c-9796644ff7f6 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:26.395916 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f3250e-0714-4bdf-919c-9796644ff7f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:26.396192 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f3250e-0714-4bdf-919c-9796644ff7f6 None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:26.396861 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f3250e-0714-4bdf-919c-9796644ff7f6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:01:26.397689 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f3250e-0714-4bdf-919c-9796644ff7f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:26.399074 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f3250e-0714-4bdf-919c-9796644ff7f6 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:26.408202 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 68/137] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Oct 12 00:01:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:26.414144 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56d8d12-2eaa-4c6a-b203-c8e0da03a0df None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:26.496776 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:26.520293 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56d8d12-2eaa-4c6a-b203-c8e0da03a0df None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:26.560718 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448f449d-b136-4ec1-ac12-d6facfdbd2a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:26.561424 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448f449d-b136-4ec1-ac12-d6facfdbd2a3 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:26.561708 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448f449d-b136-4ec1-ac12-d6facfdbd2a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:26.561985 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448f449d-b136-4ec1-ac12-d6facfdbd2a3 None admin] PATH_INFO: `/v3/users/06b6677a643746bbbbe5b869a79b1da6` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:26.563134 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448f449d-b136-4ec1-ac12-d6facfdbd2a3 None admin] RBAC: Authorizing `identity:get_user(user_id=06b6677a643746bbbbe5b869a79b1da6)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:26.564638 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448f449d-b136-4ec1-ac12-d6facfdbd2a3 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:26.566232 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 70/138] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Oct 12 00:01:26 2022] GET /identity/v3/users/06b6677a643746bbbbe5b869a79b1da6 => generated 289 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:26.572317 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f3250e-0714-4bdf-919c-9796644ff7f6 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:26.627781 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:26.680238 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f3250e-0714-4bdf-919c-9796644ff7f6 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:26.715257 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa1032e-0956-4c5a-b058-fe2ce013ae25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:26.716125 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa1032e-0956-4c5a-b058-fe2ce013ae25 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:26.716551 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa1032e-0956-4c5a-b058-fe2ce013ae25 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:26.716963 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa1032e-0956-4c5a-b058-fe2ce013ae25 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:26.787471 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa1032e-0956-4c5a-b058-fe2ce013ae25 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:26.789747 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa1032e-0956-4c5a-b058-fe2ce013ae25 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:26.839421 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:26.892810 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 69/139] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Oct 12 00:01:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 12 00:01:26.899293 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448f449d-b136-4ec1-ac12-d6facfdbd2a3 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:27.002336 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448f449d-b136-4ec1-ac12-d6facfdbd2a3 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:27.036978 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b979ad1-b4dc-4c16-86d8-b4726a57e2e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:27.038294 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b979ad1-b4dc-4c16-86d8-b4726a57e2e2 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:27.038580 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b979ad1-b4dc-4c16-86d8-b4726a57e2e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:27.038853 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b979ad1-b4dc-4c16-86d8-b4726a57e2e2 None admin] PATH_INFO: `/v3/projects/410027d6c764410ea5b76db0171903df` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:27.040111 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b979ad1-b4dc-4c16-86d8-b4726a57e2e2 None admin] RBAC: Authorizing `identity:get_project(project_id=410027d6c764410ea5b76db0171903df)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:27.041681 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b979ad1-b4dc-4c16-86d8-b4726a57e2e2 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:27.043503 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 71/140] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Oct 12 00:01:26 2022] GET /identity/v3/projects/410027d6c764410ea5b76db0171903df => generated 298 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:27.049557 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa1032e-0956-4c5a-b058-fe2ce013ae25 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:27.154907 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa1032e-0956-4c5a-b058-fe2ce013ae25 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:27.189935 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26651bc7-f5e2-4896-9b40-510de65cdad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:27.190904 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26651bc7-f5e2-4896-9b40-510de65cdad5 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:27.191402 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26651bc7-f5e2-4896-9b40-510de65cdad5 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:27.191873 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26651bc7-f5e2-4896-9b40-510de65cdad5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:27.192816 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26651bc7-f5e2-4896-9b40-510de65cdad5 None admin] RBAC: Adding query filter params (role.id=217d497bae80496a9d3772fbb90678e1, scope.project.id=410027d6c764410ea5b76db0171903df, user.id=06b6677a643746bbbbe5b869a79b1da6) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:01:27.193320 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26651bc7-f5e2-4896-9b40-510de65cdad5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:27.195004 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26651bc7-f5e2-4896-9b40-510de65cdad5 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:27.200942 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 70/141] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Oct 12 00:01:27 2022] GET /identity/v3/role_assignments?user.id=06b6677a643746bbbbe5b869a79b1da6&scope.project.id=410027d6c764410ea5b76db0171903df&role.id=217d497bae80496a9d3772fbb90678e1 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:28.199093 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87fa0c8-8c9a-44ef-b924-d71035aa9939 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:28.199678 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87fa0c8-8c9a-44ef-b924-d71035aa9939 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:28.199678 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87fa0c8-8c9a-44ef-b924-d71035aa9939 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:28.201076 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 72/142] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:01:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:01:28.207059 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0188fa23-306a-4e9b-b0e8-810c4ca247b8 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:28.207588 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0188fa23-306a-4e9b-b0e8-810c4ca247b8 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:28.207656 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0188fa23-306a-4e9b-b0e8-810c4ca247b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:28.227807 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-0188fa23-306a-4e9b-b0e8-810c4ca247b8 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:01:28.229028 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-0188fa23-306a-4e9b-b0e8-810c4ca247b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:01:28.239965 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:28.302011 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 71/143] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:01:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 00:01:28.326613 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9ed339-9b58-473b-b042-01233cb85b5e None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:28.326903 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9ed339-9b58-473b-b042-01233cb85b5e None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:28.327164 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9ed339-9b58-473b-b042-01233cb85b5e None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:28.328662 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 73/144] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:01:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:01:28.333591 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a19b1b8-e6cb-4006-9189-2c837dfabbb0 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:28.333877 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a19b1b8-e6cb-4006-9189-2c837dfabbb0 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:28.334149 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a19b1b8-e6cb-4006-9189-2c837dfabbb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:28.352512 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-6a19b1b8-e6cb-4006-9189-2c837dfabbb0 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:01:28.353678 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-6a19b1b8-e6cb-4006-9189-2c837dfabbb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:01:28.398222 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecea721d-1afc-42c3-8f9e-da7b268ad72d None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:28.398558 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecea721d-1afc-42c3-8f9e-da7b268ad72d None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:28.398782 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecea721d-1afc-42c3-8f9e-da7b268ad72d None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:28.400283 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 74/145] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:01:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 00:01:28.404837 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4264af32-4a85-4a54-bbcc-ad6506783b0b None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:28.405116 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4264af32-4a85-4a54-bbcc-ad6506783b0b None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:28.405388 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4264af32-4a85-4a54-bbcc-ad6506783b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:28.421775 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 72/146] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:01:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 00:01:28.422990 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-4264af32-4a85-4a54-bbcc-ad6506783b0b None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:01:28.424205 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-4264af32-4a85-4a54-bbcc-ad6506783b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:01:28.434141 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:28.489645 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 75/147] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:01:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 12 00:01:28.570648 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a19b1b8-e6cb-4006-9189-2c837dfabbb0 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:28.643610 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:28.667946 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a19b1b8-e6cb-4006-9189-2c837dfabbb0 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:28.685127 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4264af32-4a85-4a54-bbcc-ad6506783b0b None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:28.705366 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a954f8c8-b7bd-4201-bd9e-46206370a98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:28.706275 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a954f8c8-b7bd-4201-bd9e-46206370a98d None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:28.706554 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a954f8c8-b7bd-4201-bd9e-46206370a98d None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:28.706823 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a954f8c8-b7bd-4201-bd9e-46206370a98d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:28.718554 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a954f8c8-b7bd-4201-bd9e-46206370a98d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 00:01:28.723864 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-a954f8c8-b7bd-4201-bd9e-46206370a98d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 00:01:28.723864 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:01:28.723864 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:01:28.723864 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:01:28.723864 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:01:28.723864 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:01:28.723864 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:01:28.723864 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:01:28.723864 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:01:28.723864 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:01:28.723864 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:01:28.723864 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:01:28.723864 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 00:01:28.723864 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 00:01:28.723864 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 00:01:28.723864 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 00:01:28.725382 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 00:01:28.725382 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 00:01:28.725382 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 00:01:28.725382 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 00:01:28.725382 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:01:28.725382 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:01:28.725382 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:01:28.725382 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:01:28.725382 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:01:28.725382 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:01:28.725382 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:01:28.725382 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:01:28.725382 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:01:28.725382 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:01:28.725382 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:01:28.725382 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:01:28.726236 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:01:28.726236 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:01:28.726236 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:01:28.726236 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:01:28.726236 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 00:01:28.726236 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 00:01:28.726236 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 00:01:28.726236 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 00:01:28.726236 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 00:01:28.726236 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 00:01:28.726236 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 00:01:28.726236 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:01:28.726236 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 73/148] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Oct 12 00:01:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:01:28.730982 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a954f8c8-b7bd-4201-bd9e-46206370a98d None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:28.796167 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4264af32-4a85-4a54-bbcc-ad6506783b0b None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:28.826455 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a954f8c8-b7bd-4201-bd9e-46206370a98d None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:28.829102 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2213ac6-1025-4673-a29f-44e6eb2d8a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:28.829728 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2213ac6-1025-4673-a29f-44e6eb2d8a7f None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:28.830002 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2213ac6-1025-4673-a29f-44e6eb2d8a7f None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:28.830270 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2213ac6-1025-4673-a29f-44e6eb2d8a7f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:28.840141 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2213ac6-1025-4673-a29f-44e6eb2d8a7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 00:01:28.845043 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-c2213ac6-1025-4673-a29f-44e6eb2d8a7f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 12 00:01:28.845043 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:01:28.845043 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:01:28.845043 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:01:28.845043 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:01:28.845043 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:01:28.845043 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:01:28.845043 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:01:28.845043 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:01:28.845043 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:01:28.845043 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:01:28.845043 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:01:28.845043 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 00:01:28.845043 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 00:01:28.845043 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 00:01:28.845043 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 00:01:28.845947 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 00:01:28.845947 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 00:01:28.845947 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 00:01:28.845947 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 00:01:28.845947 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:01:28.845947 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:01:28.845947 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:01:28.845947 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:01:28.845947 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:01:28.845947 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:01:28.845947 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:01:28.845947 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:01:28.845947 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:01:28.845947 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:01:28.845947 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:01:28.845947 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:01:28.846808 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:01:28.846808 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:01:28.846808 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:01:28.846808 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:01:28.846808 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 00:01:28.846808 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 00:01:28.846808 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 00:01:28.846808 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 00:01:28.846808 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 00:01:28.846808 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 00:01:28.846808 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 12 00:01:28.846808 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:01:28.846808 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 76/149] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Oct 12 00:01:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 00:01:28.847477 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2213ac6-1025-4673-a29f-44e6eb2d8a7f None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:28.852126 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:28.861188 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9b5800-b0c2-4209-b92c-8ff77e158e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:28.861799 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b5800-b0c2-4209-b92c-8ff77e158e3a None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:28.862071 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b5800-b0c2-4209-b92c-8ff77e158e3a None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:28.862335 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b5800-b0c2-4209-b92c-8ff77e158e3a None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:28.862992 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9b5800-b0c2-4209-b92c-8ff77e158e3a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:01:28.863285 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9b5800-b0c2-4209-b92c-8ff77e158e3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:28.864697 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9b5800-b0c2-4209-b92c-8ff77e158e3a None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:28.866751 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:28.874309 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 74/150] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Oct 12 00:01:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:28.874914 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9b5800-b0c2-4209-b92c-8ff77e158e3a None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:28.947958 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2213ac6-1025-4673-a29f-44e6eb2d8a7f None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:28.970954 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9b5800-b0c2-4209-b92c-8ff77e158e3a None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:28.980978 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296f1770-15be-49eb-9f16-c1761a24ba40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:28.981608 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f1770-15be-49eb-9f16-c1761a24ba40 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:28.981890 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f1770-15be-49eb-9f16-c1761a24ba40 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:28.982153 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f1770-15be-49eb-9f16-c1761a24ba40 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:28.983262 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296f1770-15be-49eb-9f16-c1761a24ba40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:28.984724 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296f1770-15be-49eb-9f16-c1761a24ba40 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:28.986278 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 77/151] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:01:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:28.987027 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296f1770-15be-49eb-9f16-c1761a24ba40 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:29.006126 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3005863-b299-4bc4-8475-28276c68022a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:29.006741 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3005863-b299-4bc4-8475-28276c68022a None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:29.007018 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3005863-b299-4bc4-8475-28276c68022a None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:29.007288 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3005863-b299-4bc4-8475-28276c68022a None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:29.007976 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3005863-b299-4bc4-8475-28276c68022a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:01:29.008276 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3005863-b299-4bc4-8475-28276c68022a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:29.009674 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3005863-b299-4bc4-8475-28276c68022a None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:29.017487 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 75/152] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Oct 12 00:01:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:29.017487 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3005863-b299-4bc4-8475-28276c68022a None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:29.061517 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:29.073012 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:29.090621 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296f1770-15be-49eb-9f16-c1761a24ba40 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:29.114701 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3005863-b299-4bc4-8475-28276c68022a None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:29.124920 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161fc9e0-9ef9-4ba8-b17d-4017fc142484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:29.125846 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161fc9e0-9ef9-4ba8-b17d-4017fc142484 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:29.126127 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161fc9e0-9ef9-4ba8-b17d-4017fc142484 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:29.126554 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161fc9e0-9ef9-4ba8-b17d-4017fc142484 None admin] PATH_INFO: `/v3/users/04f7bff26d1349268f76f5e58f26e9d1` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:29.140155 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161fc9e0-9ef9-4ba8-b17d-4017fc142484 None admin] RBAC: Authorizing `identity:get_user(user_id=04f7bff26d1349268f76f5e58f26e9d1)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:29.141601 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161fc9e0-9ef9-4ba8-b17d-4017fc142484 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:29.143189 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 78/153] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Oct 12 00:01:28 2022] GET /identity/v3/users/04f7bff26d1349268f76f5e58f26e9d1 => generated 283 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:29.143968 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161fc9e0-9ef9-4ba8-b17d-4017fc142484 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:29.153011 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd532a4c-f1d2-420d-9019-7ca474184650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:29.153619 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd532a4c-f1d2-420d-9019-7ca474184650 None admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:29.153895 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd532a4c-f1d2-420d-9019-7ca474184650 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:29.154165 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd532a4c-f1d2-420d-9019-7ca474184650 None admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:29.154810 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd532a4c-f1d2-420d-9019-7ca474184650 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:29.156276 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd532a4c-f1d2-420d-9019-7ca474184650 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:29.228215 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 76/154] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Oct 12 00:01:29 2022] POST /identity/v3/users => generated 297 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:01:29.229031 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd532a4c-f1d2-420d-9019-7ca474184650 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:29.246441 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161fc9e0-9ef9-4ba8-b17d-4017fc142484 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:29.284308 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:29.290840 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b89d6f9-0b97-40dd-a9d3-86c5e51664fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:29.291569 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b89d6f9-0b97-40dd-a9d3-86c5e51664fb None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:29.291858 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b89d6f9-0b97-40dd-a9d3-86c5e51664fb None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:29.292126 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b89d6f9-0b97-40dd-a9d3-86c5e51664fb None admin] PATH_INFO: `/v3/users/04f7bff26d1349268f76f5e58f26e9d1` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:29.293335 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b89d6f9-0b97-40dd-a9d3-86c5e51664fb None admin] RBAC: Authorizing `identity:get_user(user_id=04f7bff26d1349268f76f5e58f26e9d1)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:29.294804 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b89d6f9-0b97-40dd-a9d3-86c5e51664fb None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:29.299404 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 79/155] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Oct 12 00:01:29 2022] GET /identity/v3/users/04f7bff26d1349268f76f5e58f26e9d1 => generated 283 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:29.302368 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b89d6f9-0b97-40dd-a9d3-86c5e51664fb None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:29.352111 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd532a4c-f1d2-420d-9019-7ca474184650 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:29.368068 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:29.391091 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b819ac-2c0f-4dff-8c4d-6b81578335a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:29.391765 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b819ac-2c0f-4dff-8c4d-6b81578335a5 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:29.392488 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b819ac-2c0f-4dff-8c4d-6b81578335a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:29.392488 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b819ac-2c0f-4dff-8c4d-6b81578335a5 None admin] PATH_INFO: `/v3/projects/25ba77a2f84b41cd99a774b93ada5e96` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:29.393443 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b819ac-2c0f-4dff-8c4d-6b81578335a5 None admin] RBAC: Authorizing `identity:get_project(project_id=25ba77a2f84b41cd99a774b93ada5e96)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:29.395790 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b819ac-2c0f-4dff-8c4d-6b81578335a5 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:29.397950 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 77/156] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Oct 12 00:01:29 2022] GET /identity/v3/projects/25ba77a2f84b41cd99a774b93ada5e96 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:29.404879 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b819ac-2c0f-4dff-8c4d-6b81578335a5 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:29.424454 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b89d6f9-0b97-40dd-a9d3-86c5e51664fb None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:29.464420 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc08e6d-34e4-4a35-bd01-32ff9fa1afc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:29.465069 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc08e6d-34e4-4a35-bd01-32ff9fa1afc6 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:29.465344 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc08e6d-34e4-4a35-bd01-32ff9fa1afc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:29.465612 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc08e6d-34e4-4a35-bd01-32ff9fa1afc6 None admin] PATH_INFO: `/v3/projects/25ba77a2f84b41cd99a774b93ada5e96` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:29.467899 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc08e6d-34e4-4a35-bd01-32ff9fa1afc6 None admin] RBAC: Authorizing `identity:get_project(project_id=25ba77a2f84b41cd99a774b93ada5e96)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:29.469348 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc08e6d-34e4-4a35-bd01-32ff9fa1afc6 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:29.471064 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 80/157] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Oct 12 00:01:29 2022] GET /identity/v3/projects/25ba77a2f84b41cd99a774b93ada5e96 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:29.478144 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc08e6d-34e4-4a35-bd01-32ff9fa1afc6 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:29.531177 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b819ac-2c0f-4dff-8c4d-6b81578335a5 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:29.561704 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:29.580423 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0e404f-874c-4ee9-81e0-6c2e88d53d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:29.581092 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e404f-874c-4ee9-81e0-6c2e88d53d33 None admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:29.581366 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e404f-874c-4ee9-81e0-6c2e88d53d33 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:29.581635 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e404f-874c-4ee9-81e0-6c2e88d53d33 None admin] PATH_INFO: `/v3/projects/25ba77a2f84b41cd99a774b93ada5e96/users/04f7bff26d1349268f76f5e58f26e9d1/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:29.583629 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0e404f-874c-4ee9-81e0-6c2e88d53d33 None admin] RBAC: Authorizing `identity:create_grant(project_id=25ba77a2f84b41cd99a774b93ada5e96, user_id=04f7bff26d1349268f76f5e58f26e9d1, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:29.585121 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0e404f-874c-4ee9-81e0-6c2e88d53d33 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:29.602515 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc08e6d-34e4-4a35-bd01-32ff9fa1afc6 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:29.637390 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd52b85-95e3-44f5-86f8-0684d70a63c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:29.638392 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd52b85-95e3-44f5-86f8-0684d70a63c6 None admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:29.638862 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd52b85-95e3-44f5-86f8-0684d70a63c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:29.639324 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd52b85-95e3-44f5-86f8-0684d70a63c6 None admin] PATH_INFO: `/v3/projects/25ba77a2f84b41cd99a774b93ada5e96/users/04f7bff26d1349268f76f5e58f26e9d1/roles/81991615ac0b46818426bf37015df930` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:29.642362 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd52b85-95e3-44f5-86f8-0684d70a63c6 None admin] RBAC: Authorizing `identity:create_grant(project_id=25ba77a2f84b41cd99a774b93ada5e96, user_id=04f7bff26d1349268f76f5e58f26e9d1, role_id=81991615ac0b46818426bf37015df930)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:29.644807 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd52b85-95e3-44f5-86f8-0684d70a63c6 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:29.646655 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 78/158] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Oct 12 00:01:29 2022] PUT /identity/v3/projects/25ba77a2f84b41cd99a774b93ada5e96/users/04f7bff26d1349268f76f5e58f26e9d1/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:01:29.742877 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 81/159] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Oct 12 00:01:29 2022] PUT /identity/v3/projects/25ba77a2f84b41cd99a774b93ada5e96/users/04f7bff26d1349268f76f5e58f26e9d1/roles/81991615ac0b46818426bf37015df930 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 00:01:31.501208 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64275038-292d-4bfb-80a8-7d5df08f55dd None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:31.502255 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64275038-292d-4bfb-80a8-7d5df08f55dd None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:31.502736 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64275038-292d-4bfb-80a8-7d5df08f55dd None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:31.504465 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 79/160] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:01:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:01:31.511020 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d55ca6-45b0-4d21-95fd-6e269442a3f8 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:31.512109 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d55ca6-45b0-4d21-95fd-6e269442a3f8 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:31.512583 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d55ca6-45b0-4d21-95fd-6e269442a3f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:31.531489 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-b6d55ca6-45b0-4d21-95fd-6e269442a3f8 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:01:31.533057 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-b6d55ca6-45b0-4d21-95fd-6e269442a3f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:01:31.597831 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9590d48-57fd-4d4e-a8a1-e054a865e44e None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:31.598160 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9590d48-57fd-4d4e-a8a1-e054a865e44e None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:31.598388 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9590d48-57fd-4d4e-a8a1-e054a865e44e None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:31.600366 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 80/161] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:01:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:01:31.605370 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8686e3c-6b63-4ff2-af29-864f990842ef None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:31.605648 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8686e3c-6b63-4ff2-af29-864f990842ef None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:31.605925 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8686e3c-6b63-4ff2-af29-864f990842ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:31.618718 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 82/162] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:01:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 00:01:31.625219 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-a8686e3c-6b63-4ff2-af29-864f990842ef None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:01:31.626665 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-a8686e3c-6b63-4ff2-af29-864f990842ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:01:31.664964 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:31.701359 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 81/163] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:01:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 00:01:31.811114 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53164405-3bed-4b59-845b-9674c0176085 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:31.811816 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53164405-3bed-4b59-845b-9674c0176085 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:31.812310 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53164405-3bed-4b59-845b-9674c0176085 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:31.814046 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 83/164] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:01:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:01:31.820212 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46ab2f6-ac88-49c9-b6f4-0907527b8a48 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:31.820873 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46ab2f6-ac88-49c9-b6f4-0907527b8a48 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:31.821353 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46ab2f6-ac88-49c9-b6f4-0907527b8a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:31.839774 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-b46ab2f6-ac88-49c9-b6f4-0907527b8a48 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:01:31.841274 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-b46ab2f6-ac88-49c9-b6f4-0907527b8a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:01:31.905799 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53164405-3bed-4b59-845b-9674c0176085 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:31.912759 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 82/165] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:01:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 00:01:31.930676 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c090f4c2-bf6d-492d-828c-6dd67e230a26 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:31.930965 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c090f4c2-bf6d-492d-828c-6dd67e230a26 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:31.931257 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c090f4c2-bf6d-492d-828c-6dd67e230a26 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:31.932615 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 83/166] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:01:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 00:01:31.937748 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c237b5d-62b7-4fb6-bdb1-67413c2d0a7a None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:31.938030 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c237b5d-62b7-4fb6-bdb1-67413c2d0a7a None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:31.938301 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c237b5d-62b7-4fb6-bdb1-67413c2d0a7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:31.958289 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-2c237b5d-62b7-4fb6-bdb1-67413c2d0a7a None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:01:31.959559 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-2c237b5d-62b7-4fb6-bdb1-67413c2d0a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:01:31.971768 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:32.014215 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53164405-3bed-4b59-845b-9674c0176085 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:32.032542 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 84/167] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:01:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 12 00:01:32.033653 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c237b5d-62b7-4fb6-bdb1-67413c2d0a7a None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:32.059967 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0341ec-0bf5-4608-a6eb-b6fccfc09988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:32.061026 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0341ec-0bf5-4608-a6eb-b6fccfc09988 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:32.061459 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0341ec-0bf5-4608-a6eb-b6fccfc09988 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:32.061776 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0341ec-0bf5-4608-a6eb-b6fccfc09988 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:32.066839 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:32.074273 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0341ec-0bf5-4608-a6eb-b6fccfc09988 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 00:01:32.079536 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-9c0341ec-0bf5-4608-a6eb-b6fccfc09988 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 12 00:01:32.079536 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:01:32.079536 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:01:32.079536 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:01:32.079536 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:01:32.079536 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:01:32.079536 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:01:32.079536 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:01:32.079536 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:01:32.079536 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:01:32.079536 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:01:32.079536 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:01:32.079536 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 00:01:32.079536 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 00:01:32.079536 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 00:01:32.079536 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 00:01:32.080582 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 00:01:32.080582 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 00:01:32.080582 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 00:01:32.080582 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 00:01:32.080582 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:01:32.080582 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:01:32.080582 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:01:32.080582 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:01:32.080582 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:01:32.080582 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:01:32.080582 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:01:32.080582 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:01:32.080582 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:01:32.080582 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:01:32.080582 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:01:32.080582 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:01:32.081409 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:01:32.081409 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:01:32.081409 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:01:32.081409 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:01:32.081409 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 00:01:32.081409 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 00:01:32.081409 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 00:01:32.081409 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 00:01:32.081409 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 00:01:32.081409 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 00:01:32.081409 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 12 00:01:32.081409 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:01:32.086303 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 84/168] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Oct 12 00:01:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 00:01:32.087335 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0341ec-0bf5-4608-a6eb-b6fccfc09988 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:32.143230 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c237b5d-62b7-4fb6-bdb1-67413c2d0a7a None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:32.181782 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f644664-5dce-4b75-97b2-e21ba13b16b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:32.182802 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f644664-5dce-4b75-97b2-e21ba13b16b6 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:32.183292 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f644664-5dce-4b75-97b2-e21ba13b16b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:32.183804 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f644664-5dce-4b75-97b2-e21ba13b16b6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:32.185182 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f644664-5dce-4b75-97b2-e21ba13b16b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:32.186843 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f644664-5dce-4b75-97b2-e21ba13b16b6 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:32.188694 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 85/169] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:01:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:32.194172 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f644664-5dce-4b75-97b2-e21ba13b16b6 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:32.206403 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0341ec-0bf5-4608-a6eb-b6fccfc09988 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:32.224259 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:32.244475 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60a782c-7dd6-457e-8ff6-84bf1a7fb3a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:32.245087 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a782c-7dd6-457e-8ff6-84bf1a7fb3a4 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:32.245363 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a782c-7dd6-457e-8ff6-84bf1a7fb3a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:32.245636 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a782c-7dd6-457e-8ff6-84bf1a7fb3a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:32.246299 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60a782c-7dd6-457e-8ff6-84bf1a7fb3a4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:01:32.246601 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60a782c-7dd6-457e-8ff6-84bf1a7fb3a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:32.248028 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60a782c-7dd6-457e-8ff6-84bf1a7fb3a4 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:32.256765 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 85/170] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Oct 12 00:01:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:32.257877 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60a782c-7dd6-457e-8ff6-84bf1a7fb3a4 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:32.303038 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:32.306154 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f644664-5dce-4b75-97b2-e21ba13b16b6 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:32.340267 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8d5de7-0c84-4659-ae12-8189d5b7e858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:32.341035 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d5de7-0c84-4659-ae12-8189d5b7e858 None admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:32.341517 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d5de7-0c84-4659-ae12-8189d5b7e858 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:32.341809 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d5de7-0c84-4659-ae12-8189d5b7e858 None admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:32.342779 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8d5de7-0c84-4659-ae12-8189d5b7e858 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:32.344259 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8d5de7-0c84-4659-ae12-8189d5b7e858 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:32.360502 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60a782c-7dd6-457e-8ff6-84bf1a7fb3a4 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:32.392607 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358ba550-801f-4dc1-9eed-c827e3edcd09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:32.393235 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358ba550-801f-4dc1-9eed-c827e3edcd09 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:32.393514 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358ba550-801f-4dc1-9eed-c827e3edcd09 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:32.393783 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358ba550-801f-4dc1-9eed-c827e3edcd09 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:32.404283 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358ba550-801f-4dc1-9eed-c827e3edcd09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 00:01:32.409638 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-358ba550-801f-4dc1-9eed-c827e3edcd09 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 00:01:32.409638 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:01:32.409638 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:01:32.409638 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:01:32.409638 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:01:32.409638 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:01:32.409638 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:01:32.409638 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:01:32.409638 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:01:32.409638 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:01:32.409638 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:01:32.409638 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:01:32.409638 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 00:01:32.409638 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 00:01:32.409638 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 00:01:32.409638 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 00:01:32.410711 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 00:01:32.410711 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 00:01:32.410711 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 00:01:32.410711 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 00:01:32.410711 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:01:32.410711 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:01:32.410711 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:01:32.410711 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:01:32.410711 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:01:32.410711 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:01:32.410711 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:01:32.410711 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:01:32.410711 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:01:32.410711 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:01:32.410711 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:01:32.410711 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:01:32.411656 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:01:32.411656 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:01:32.411656 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:01:32.411656 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:01:32.411656 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 00:01:32.411656 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 00:01:32.411656 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 00:01:32.411656 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 00:01:32.411656 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 00:01:32.411656 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 00:01:32.411656 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 00:01:32.411656 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:01:32.411656 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 86/171] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Oct 12 00:01:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:01:32.412937 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358ba550-801f-4dc1-9eed-c827e3edcd09 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:32.449014 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 86/172] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Oct 12 00:01:32 2022] POST /identity/v3/users => generated 297 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:01:32.449787 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:32.449899 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8d5de7-0c84-4659-ae12-8189d5b7e858 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:32.516585 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358ba550-801f-4dc1-9eed-c827e3edcd09 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:32.560937 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285cbe3a-7c36-43ff-b3e6-0c956e3c82d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:32.561590 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285cbe3a-7c36-43ff-b3e6-0c956e3c82d8 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:32.561886 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285cbe3a-7c36-43ff-b3e6-0c956e3c82d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:32.562156 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285cbe3a-7c36-43ff-b3e6-0c956e3c82d8 None admin] PATH_INFO: `/v3/users/4ef91a6924f74989bb81ed49abf61a4d` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:32.576758 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8d5de7-0c84-4659-ae12-8189d5b7e858 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:32.578739 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285cbe3a-7c36-43ff-b3e6-0c956e3c82d8 None admin] RBAC: Authorizing `identity:get_user(user_id=4ef91a6924f74989bb81ed49abf61a4d)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:32.580391 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285cbe3a-7c36-43ff-b3e6-0c956e3c82d8 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:32.582309 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 87/173] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Oct 12 00:01:32 2022] GET /identity/v3/users/4ef91a6924f74989bb81ed49abf61a4d => generated 297 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:32.583059 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285cbe3a-7c36-43ff-b3e6-0c956e3c82d8 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:32.618143 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215f6e18-1941-436d-9241-fe78cb7df13a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:32.618703 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f6e18-1941-436d-9241-fe78cb7df13a None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:32.618979 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f6e18-1941-436d-9241-fe78cb7df13a None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:32.619255 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f6e18-1941-436d-9241-fe78cb7df13a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:32.630488 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215f6e18-1941-436d-9241-fe78cb7df13a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 00:01:32.636674 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-215f6e18-1941-436d-9241-fe78cb7df13a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 12 00:01:32.636674 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:01:32.636674 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:01:32.636674 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:01:32.636674 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:01:32.636674 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:01:32.636674 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:01:32.636674 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:01:32.636674 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:01:32.636674 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:01:32.636674 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:01:32.636674 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:01:32.636674 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 00:01:32.636674 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 00:01:32.636674 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 00:01:32.636674 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 00:01:32.637533 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 00:01:32.637533 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 00:01:32.637533 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 00:01:32.637533 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 00:01:32.637533 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:01:32.637533 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:01:32.637533 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:01:32.637533 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:01:32.637533 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:01:32.637533 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:01:32.637533 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:01:32.637533 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:01:32.637533 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:01:32.637533 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:01:32.637533 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:01:32.637533 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:01:32.638383 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:01:32.638383 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:01:32.638383 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:01:32.638383 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:01:32.638383 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 00:01:32.638383 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 00:01:32.638383 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 00:01:32.638383 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 00:01:32.638383 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 00:01:32.638383 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 00:01:32.638383 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 12 00:01:32.638383 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:01:32.638383 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 87/174] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Oct 12 00:01:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:01:32.640105 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215f6e18-1941-436d-9241-fe78cb7df13a None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:32.665620 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:32.695295 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:32.699906 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285cbe3a-7c36-43ff-b3e6-0c956e3c82d8 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:32.740701 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bbc533-caa3-416f-a663-f95bcc6f28d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:32.741324 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bbc533-caa3-416f-a663-f95bcc6f28d6 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:32.741761 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bbc533-caa3-416f-a663-f95bcc6f28d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:32.742032 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bbc533-caa3-416f-a663-f95bcc6f28d6 None admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:32.743022 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bbc533-caa3-416f-a663-f95bcc6f28d6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:01:32.743323 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bbc533-caa3-416f-a663-f95bcc6f28d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:32.745046 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bbc533-caa3-416f-a663-f95bcc6f28d6 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:32.753558 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215f6e18-1941-436d-9241-fe78cb7df13a None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:32.758024 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 88/175] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Oct 12 00:01:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:32.759084 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bbc533-caa3-416f-a663-f95bcc6f28d6 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:32.802789 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c3c527-c483-44ae-825f-0bc789434a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:32.803506 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c3c527-c483-44ae-825f-0bc789434a8e None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:32.803782 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c3c527-c483-44ae-825f-0bc789434a8e None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:32.804050 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c3c527-c483-44ae-825f-0bc789434a8e None admin] PATH_INFO: `/v3/projects/25ba77a2f84b41cd99a774b93ada5e96` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:32.805512 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c3c527-c483-44ae-825f-0bc789434a8e None admin] RBAC: Authorizing `identity:get_project(project_id=25ba77a2f84b41cd99a774b93ada5e96)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:32.806960 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c3c527-c483-44ae-825f-0bc789434a8e None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:32.808800 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 88/176] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Oct 12 00:01:32 2022] GET /identity/v3/projects/25ba77a2f84b41cd99a774b93ada5e96 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:32.809518 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c3c527-c483-44ae-825f-0bc789434a8e None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:32.883997 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bbc533-caa3-416f-a663-f95bcc6f28d6 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:32.911459 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:32.928563 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d1c154-c61e-4320-8651-00cd3b3ddb4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:32.929499 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d1c154-c61e-4320-8651-00cd3b3ddb4c None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:32.929949 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d1c154-c61e-4320-8651-00cd3b3ddb4c None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:32.930392 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d1c154-c61e-4320-8651-00cd3b3ddb4c None admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:32.931224 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d1c154-c61e-4320-8651-00cd3b3ddb4c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:01:32.931353 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c3c527-c483-44ae-825f-0bc789434a8e None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:32.932165 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d1c154-c61e-4320-8651-00cd3b3ddb4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:32.935680 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d1c154-c61e-4320-8651-00cd3b3ddb4c None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:32.947226 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 89/177] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Oct 12 00:01:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:32.948304 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d1c154-c61e-4320-8651-00cd3b3ddb4c None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:32.976451 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:32.981936 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcd9da3-379b-43e5-b81a-af02814ed372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:32.982833 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcd9da3-379b-43e5-b81a-af02814ed372 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:32.983182 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcd9da3-379b-43e5-b81a-af02814ed372 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:32.983574 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcd9da3-379b-43e5-b81a-af02814ed372 None admin] PATH_INFO: `/v3/users/04f7bff26d1349268f76f5e58f26e9d1` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:32.985510 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcd9da3-379b-43e5-b81a-af02814ed372 None admin] RBAC: Authorizing `identity:get_user(user_id=04f7bff26d1349268f76f5e58f26e9d1)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:32.987542 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcd9da3-379b-43e5-b81a-af02814ed372 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:32.989554 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 89/178] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Oct 12 00:01:32 2022] GET /identity/v3/users/04f7bff26d1349268f76f5e58f26e9d1 => generated 283 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:32.991605 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcd9da3-379b-43e5-b81a-af02814ed372 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:33.073230 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d1c154-c61e-4320-8651-00cd3b3ddb4c None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:33.110632 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8723828-0847-428f-8117-013ff48d530e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:33.111504 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8723828-0847-428f-8117-013ff48d530e None admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:33.111882 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8723828-0847-428f-8117-013ff48d530e None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:33.112238 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8723828-0847-428f-8117-013ff48d530e None admin] PATH_INFO: `/v3/projects/25ba77a2f84b41cd99a774b93ada5e96/users/4ef91a6924f74989bb81ed49abf61a4d/roles/ef689f96442945e8a7bc8cbfee13d909` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:33.114557 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8723828-0847-428f-8117-013ff48d530e None admin] RBAC: Authorizing `identity:create_grant(project_id=25ba77a2f84b41cd99a774b93ada5e96, user_id=4ef91a6924f74989bb81ed49abf61a4d, role_id=ef689f96442945e8a7bc8cbfee13d909)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:33.116511 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8723828-0847-428f-8117-013ff48d530e None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:33.132410 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcd9da3-379b-43e5-b81a-af02814ed372 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:33.167474 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45796913-5e60-4daf-8891-0a16cb53f83f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:33.168252 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45796913-5e60-4daf-8891-0a16cb53f83f None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:33.168459 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45796913-5e60-4daf-8891-0a16cb53f83f None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:33.168728 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45796913-5e60-4daf-8891-0a16cb53f83f None admin] PATH_INFO: `/v3/users/04f7bff26d1349268f76f5e58f26e9d1` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:33.169844 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45796913-5e60-4daf-8891-0a16cb53f83f None admin] RBAC: Authorizing `identity:get_user(user_id=04f7bff26d1349268f76f5e58f26e9d1)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:33.171288 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45796913-5e60-4daf-8891-0a16cb53f83f None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:33.177480 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 90/179] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Oct 12 00:01:32 2022] GET /identity/v3/users/04f7bff26d1349268f76f5e58f26e9d1 => generated 283 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:33.177480 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45796913-5e60-4daf-8891-0a16cb53f83f None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:33.206093 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 90/180] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Oct 12 00:01:32 2022] PUT /identity/v3/projects/25ba77a2f84b41cd99a774b93ada5e96/users/4ef91a6924f74989bb81ed49abf61a4d/roles/ef689f96442945e8a7bc8cbfee13d909 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:01:33.206713 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8723828-0847-428f-8117-013ff48d530e None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:33.208886 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:33.319730 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:33.353019 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8723828-0847-428f-8117-013ff48d530e None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:33.377058 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45796913-5e60-4daf-8891-0a16cb53f83f None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:33.420201 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddb28ed-721b-4f7e-9ba0-d6eaab6991d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:33.421357 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddb28ed-721b-4f7e-9ba0-d6eaab6991d2 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:33.422002 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddb28ed-721b-4f7e-9ba0-d6eaab6991d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:33.422495 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddb28ed-721b-4f7e-9ba0-d6eaab6991d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:33.428880 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117725d7-9dc0-4b81-8b79-c626d9196dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:33.429504 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117725d7-9dc0-4b81-8b79-c626d9196dd8 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:33.429778 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117725d7-9dc0-4b81-8b79-c626d9196dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:33.430049 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117725d7-9dc0-4b81-8b79-c626d9196dd8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:33.502979 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddb28ed-721b-4f7e-9ba0-d6eaab6991d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:33.504490 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddb28ed-721b-4f7e-9ba0-d6eaab6991d2 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:33.511405 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117725d7-9dc0-4b81-8b79-c626d9196dd8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:33.513162 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117725d7-9dc0-4b81-8b79-c626d9196dd8 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:33.532040 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:33.626410 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:33.629058 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 91/181] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Oct 12 00:01:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 12 00:01:33.635914 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddb28ed-721b-4f7e-9ba0-d6eaab6991d2 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:33.658380 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 91/182] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Oct 12 00:01:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 12 00:01:33.676740 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117725d7-9dc0-4b81-8b79-c626d9196dd8 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:33.801876 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddb28ed-721b-4f7e-9ba0-d6eaab6991d2 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:33.825618 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:33.842642 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7275387-66ad-429e-9d08-10c3679f9687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:33.843284 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7275387-66ad-429e-9d08-10c3679f9687 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:33.843582 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7275387-66ad-429e-9d08-10c3679f9687 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:33.843848 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7275387-66ad-429e-9d08-10c3679f9687 None admin] PATH_INFO: `/v3/projects/25ba77a2f84b41cd99a774b93ada5e96` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:33.844967 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7275387-66ad-429e-9d08-10c3679f9687 None admin] RBAC: Authorizing `identity:get_project(project_id=25ba77a2f84b41cd99a774b93ada5e96)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:33.846911 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7275387-66ad-429e-9d08-10c3679f9687 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:33.849374 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 92/183] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Oct 12 00:01:33 2022] GET /identity/v3/projects/25ba77a2f84b41cd99a774b93ada5e96 => generated 302 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:33.856342 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7275387-66ad-429e-9d08-10c3679f9687 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:33.896995 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117725d7-9dc0-4b81-8b79-c626d9196dd8 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:33.958149 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538c17c9-92e4-4c3e-9e6d-9a1818dd40c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:33.959555 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538c17c9-92e4-4c3e-9e6d-9a1818dd40c9 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:33.959983 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538c17c9-92e4-4c3e-9e6d-9a1818dd40c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:33.960388 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538c17c9-92e4-4c3e-9e6d-9a1818dd40c9 None admin] PATH_INFO: `/v3/projects/25ba77a2f84b41cd99a774b93ada5e96` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:33.961770 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538c17c9-92e4-4c3e-9e6d-9a1818dd40c9 None admin] RBAC: Authorizing `identity:get_project(project_id=25ba77a2f84b41cd99a774b93ada5e96)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:33.963538 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538c17c9-92e4-4c3e-9e6d-9a1818dd40c9 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:33.966416 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 92/184] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Oct 12 00:01:33 2022] GET /identity/v3/projects/25ba77a2f84b41cd99a774b93ada5e96 => generated 302 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 00:01:33.972881 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538c17c9-92e4-4c3e-9e6d-9a1818dd40c9 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:34.030826 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7275387-66ad-429e-9d08-10c3679f9687 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:34.059161 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:34.093197 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9559af78-8baf-4b55-b885-7e48abe6b035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:34.093814 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9559af78-8baf-4b55-b885-7e48abe6b035 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:34.094083 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9559af78-8baf-4b55-b885-7e48abe6b035 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:34.094348 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9559af78-8baf-4b55-b885-7e48abe6b035 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:34.095073 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9559af78-8baf-4b55-b885-7e48abe6b035 None admin] RBAC: Adding query filter params (role.id=b2e41f0d7a304dc7af8b6a232049b6a2, scope.project.id=25ba77a2f84b41cd99a774b93ada5e96, user.id=04f7bff26d1349268f76f5e58f26e9d1) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:01:34.095386 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9559af78-8baf-4b55-b885-7e48abe6b035 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:34.098042 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9559af78-8baf-4b55-b885-7e48abe6b035 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:34.110984 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 93/185] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Oct 12 00:01:33 2022] GET /identity/v3/role_assignments?user.id=04f7bff26d1349268f76f5e58f26e9d1&scope.project.id=25ba77a2f84b41cd99a774b93ada5e96&role.id=b2e41f0d7a304dc7af8b6a232049b6a2 => generated 608 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:34.142028 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538c17c9-92e4-4c3e-9e6d-9a1818dd40c9 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:34.190716 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189ee496-59c3-4198-8df2-f0b75a209093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:34.191338 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189ee496-59c3-4198-8df2-f0b75a209093 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:34.191698 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189ee496-59c3-4198-8df2-f0b75a209093 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:34.191969 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189ee496-59c3-4198-8df2-f0b75a209093 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:34.192701 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189ee496-59c3-4198-8df2-f0b75a209093 None admin] RBAC: Adding query filter params (role.id=81991615ac0b46818426bf37015df930, scope.project.id=25ba77a2f84b41cd99a774b93ada5e96, user.id=04f7bff26d1349268f76f5e58f26e9d1) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:01:34.193001 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189ee496-59c3-4198-8df2-f0b75a209093 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:34.194562 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189ee496-59c3-4198-8df2-f0b75a209093 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:34.204582 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 93/186] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Oct 12 00:01:33 2022] GET /identity/v3/role_assignments?user.id=04f7bff26d1349268f76f5e58f26e9d1&scope.project.id=25ba77a2f84b41cd99a774b93ada5e96&role.id=81991615ac0b46818426bf37015df930 => generated 608 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 00:01:34.958692 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a3dd2d-c24f-47d3-bf4e-a9a719fa96ce None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:34.959844 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a3dd2d-c24f-47d3-bf4e-a9a719fa96ce None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:34.959844 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a3dd2d-c24f-47d3-bf4e-a9a719fa96ce None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:34.961199 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 94/187] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:01:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:01:34.966266 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1befbffa-f63f-46fa-a31d-faf4832e65c7 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:34.966652 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1befbffa-f63f-46fa-a31d-faf4832e65c7 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:34.966805 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1befbffa-f63f-46fa-a31d-faf4832e65c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:34.985080 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-1befbffa-f63f-46fa-a31d-faf4832e65c7 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:01:34.986263 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-1befbffa-f63f-46fa-a31d-faf4832e65c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:01:35.003637 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a90ac-4daa-4341-a7ce-ea805212e4b3 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:35.003917 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a90ac-4daa-4341-a7ce-ea805212e4b3 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:35.004183 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a90ac-4daa-4341-a7ce-ea805212e4b3 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:35.005517 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 95/188] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:01:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 00:01:35.010618 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f327b4-0dd8-46d5-830f-347b414fa809 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:35.010897 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f327b4-0dd8-46d5-830f-347b414fa809 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:35.011165 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f327b4-0dd8-46d5-830f-347b414fa809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:35.029659 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-28f327b4-0dd8-46d5-830f-347b414fa809 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:01:35.030836 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-28f327b4-0dd8-46d5-830f-347b414fa809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:01:35.059531 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 94/189] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:01:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 00:01:35.079544 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:35.099141 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 96/190] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:01:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 12 00:01:35.326881 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1befbffa-f63f-46fa-a31d-faf4832e65c7 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:35.366776 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f327b4-0dd8-46d5-830f-347b414fa809 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:35.371803 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:35.432027 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1befbffa-f63f-46fa-a31d-faf4832e65c7 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:35.466204 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a989b72-cf0a-48ce-baf7-24a4e78a20dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:35.466836 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a989b72-cf0a-48ce-baf7-24a4e78a20dc None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:35.467110 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a989b72-cf0a-48ce-baf7-24a4e78a20dc None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:35.467397 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a989b72-cf0a-48ce-baf7-24a4e78a20dc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:35.467671 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f327b4-0dd8-46d5-830f-347b414fa809 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:35.468480 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a989b72-cf0a-48ce-baf7-24a4e78a20dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:35.469908 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a989b72-cf0a-48ce-baf7-24a4e78a20dc None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:35.471690 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 95/191] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:01:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:35.477404 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a989b72-cf0a-48ce-baf7-24a4e78a20dc None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:35.507572 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deb2bf6-82ff-40a2-913e-de951dacb346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:35.508223 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb2bf6-82ff-40a2-913e-de951dacb346 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:35.508768 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb2bf6-82ff-40a2-913e-de951dacb346 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:35.508768 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb2bf6-82ff-40a2-913e-de951dacb346 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:35.520963 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deb2bf6-82ff-40a2-913e-de951dacb346 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 00:01:35.526573 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-2deb2bf6-82ff-40a2-913e-de951dacb346 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 12 00:01:35.526573 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:01:35.526573 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:01:35.526573 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:01:35.526573 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:01:35.526573 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:01:35.526573 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:01:35.526573 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:01:35.526573 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:01:35.526573 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:01:35.526573 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:01:35.526573 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:01:35.526573 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 00:01:35.526573 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 00:01:35.526573 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 00:01:35.526573 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 00:01:35.527601 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 00:01:35.527601 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 00:01:35.527601 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 00:01:35.527601 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 00:01:35.527601 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:01:35.527601 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:01:35.527601 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:01:35.527601 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:01:35.527601 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:01:35.527601 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:01:35.527601 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:01:35.527601 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:01:35.527601 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:01:35.527601 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:01:35.527601 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:01:35.527601 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:01:35.528563 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:01:35.528563 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:01:35.528563 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:01:35.528563 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:01:35.528563 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 00:01:35.528563 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 00:01:35.528563 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 00:01:35.528563 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 00:01:35.528563 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 00:01:35.528563 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 00:01:35.528563 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 12 00:01:35.528563 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:01:35.528563 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 97/192] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Oct 12 00:01:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 00:01:35.533740 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deb2bf6-82ff-40a2-913e-de951dacb346 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:35.582310 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:35.598957 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a989b72-cf0a-48ce-baf7-24a4e78a20dc None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:35.609875 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:35.635792 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c41d62-e358-4430-8a44-967dba636a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:35.636394 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c41d62-e358-4430-8a44-967dba636a88 None admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:35.636668 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c41d62-e358-4430-8a44-967dba636a88 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:35.636940 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c41d62-e358-4430-8a44-967dba636a88 None admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:35.637587 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c41d62-e358-4430-8a44-967dba636a88 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:35.639029 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c41d62-e358-4430-8a44-967dba636a88 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:35.640701 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deb2bf6-82ff-40a2-913e-de951dacb346 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:35.674152 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4f2dc4-4286-430e-b3bd-90c51328d543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:35.674753 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4f2dc4-4286-430e-b3bd-90c51328d543 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:35.675028 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4f2dc4-4286-430e-b3bd-90c51328d543 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:35.675297 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4f2dc4-4286-430e-b3bd-90c51328d543 None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:35.675984 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4f2dc4-4286-430e-b3bd-90c51328d543 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:01:35.676282 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4f2dc4-4286-430e-b3bd-90c51328d543 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:35.677695 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4f2dc4-4286-430e-b3bd-90c51328d543 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:35.685510 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 98/193] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Oct 12 00:01:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 00:01:35.686632 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8334a6-7a45-455c-99e3-26c0273b8418 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:35.686893 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8334a6-7a45-455c-99e3-26c0273b8418 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:35.687169 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8334a6-7a45-455c-99e3-26c0273b8418 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:35.688522 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 99/194] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:01:35 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:01:35.691301 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8334a6-7a45-455c-99e3-26c0273b8418 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:35.731011 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 96/195] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Oct 12 00:01:35 2022] POST /identity/v3/users => generated 293 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 00:01:35.736945 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec52376a-a709-4d99-9bc6-26f92dc2ae46 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:35.737232 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec52376a-a709-4d99-9bc6-26f92dc2ae46 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:35.737500 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec52376a-a709-4d99-9bc6-26f92dc2ae46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:35.759253 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-ec52376a-a709-4d99-9bc6-26f92dc2ae46 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:01:35.760776 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-ec52376a-a709-4d99-9bc6-26f92dc2ae46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:01:35.802248 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8334a6-7a45-455c-99e3-26c0273b8418 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:35.806162 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:35.839231 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 97/196] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:01:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 00:01:35.846364 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d65402c-312a-4fdf-bb77-547a5bee0cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:35.847021 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d65402c-312a-4fdf-bb77-547a5bee0cd1 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:35.847299 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d65402c-312a-4fdf-bb77-547a5bee0cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:35.847607 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d65402c-312a-4fdf-bb77-547a5bee0cd1 None admin] PATH_INFO: `/v3/users/d92470f780ec4fc3becaebe7ea052bce` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:35.862987 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d65402c-312a-4fdf-bb77-547a5bee0cd1 None admin] RBAC: Authorizing `identity:get_user(user_id=d92470f780ec4fc3becaebe7ea052bce)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:35.864498 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d65402c-312a-4fdf-bb77-547a5bee0cd1 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:35.866959 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 100/197] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Oct 12 00:01:35 2022] GET /identity/v3/users/d92470f780ec4fc3becaebe7ea052bce => generated 297 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 00:01:35.872514 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec52376a-a709-4d99-9bc6-26f92dc2ae46 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:35.971625 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:35.978599 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec52376a-a709-4d99-9bc6-26f92dc2ae46 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:36.016816 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f270e0d0-c15f-4ad6-84e3-11a2d96260f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:36.017452 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f270e0d0-c15f-4ad6-84e3-11a2d96260f7 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:36.017728 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f270e0d0-c15f-4ad6-84e3-11a2d96260f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:36.017997 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f270e0d0-c15f-4ad6-84e3-11a2d96260f7 None admin] PATH_INFO: `/v3/projects/25ba77a2f84b41cd99a774b93ada5e96` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:36.019133 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f270e0d0-c15f-4ad6-84e3-11a2d96260f7 None admin] RBAC: Authorizing `identity:get_project(project_id=25ba77a2f84b41cd99a774b93ada5e96)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:36.020613 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f270e0d0-c15f-4ad6-84e3-11a2d96260f7 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:36.022686 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 98/198] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Oct 12 00:01:35 2022] GET /identity/v3/projects/25ba77a2f84b41cd99a774b93ada5e96 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:36.028284 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d65402c-312a-4fdf-bb77-547a5bee0cd1 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:36.103721 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f270e0d0-c15f-4ad6-84e3-11a2d96260f7 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:36.140380 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d65402c-312a-4fdf-bb77-547a5bee0cd1 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:36.178331 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226260d3-331e-4af5-aff2-ec6875d00e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:36.179452 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226260d3-331e-4af5-aff2-ec6875d00e1e None admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:36.179873 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226260d3-331e-4af5-aff2-ec6875d00e1e None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:36.180275 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226260d3-331e-4af5-aff2-ec6875d00e1e None admin] PATH_INFO: `/v3/projects/25ba77a2f84b41cd99a774b93ada5e96/users/d92470f780ec4fc3becaebe7ea052bce/roles/217d497bae80496a9d3772fbb90678e1` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:36.183159 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226260d3-331e-4af5-aff2-ec6875d00e1e None admin] RBAC: Authorizing `identity:create_grant(project_id=25ba77a2f84b41cd99a774b93ada5e96, user_id=d92470f780ec4fc3becaebe7ea052bce, role_id=217d497bae80496a9d3772fbb90678e1)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:36.185459 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226260d3-331e-4af5-aff2-ec6875d00e1e None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:36.225168 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f270e0d0-c15f-4ad6-84e3-11a2d96260f7 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:36.257889 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 101/199] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Oct 12 00:01:36 2022] PUT /identity/v3/projects/25ba77a2f84b41cd99a774b93ada5e96/users/d92470f780ec4fc3becaebe7ea052bce/roles/217d497bae80496a9d3772fbb90678e1 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:01:36.271404 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02558340-7a39-4a1b-938b-de16894fd464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:36.271404 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02558340-7a39-4a1b-938b-de16894fd464 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:36.271404 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02558340-7a39-4a1b-938b-de16894fd464 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:36.271404 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02558340-7a39-4a1b-938b-de16894fd464 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:36.283564 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02558340-7a39-4a1b-938b-de16894fd464 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 00:01:36.289553 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-02558340-7a39-4a1b-938b-de16894fd464 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 12 00:01:36.289553 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:01:36.289553 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:01:36.289553 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:01:36.289553 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:01:36.289553 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:01:36.289553 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:01:36.289553 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:01:36.289553 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:01:36.289553 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:01:36.289553 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:01:36.289553 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:01:36.289553 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 00:01:36.289553 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 00:01:36.289553 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 00:01:36.289553 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 00:01:36.290705 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 00:01:36.290705 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 00:01:36.290705 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 00:01:36.290705 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 00:01:36.290705 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:01:36.290705 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:01:36.290705 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:01:36.290705 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:01:36.290705 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:01:36.290705 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:01:36.290705 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:01:36.290705 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:01:36.290705 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:01:36.290705 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:01:36.290705 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:01:36.290705 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:01:36.291997 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:01:36.291997 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:01:36.291997 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:01:36.291997 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:01:36.291997 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 00:01:36.291997 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 00:01:36.291997 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 00:01:36.291997 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 00:01:36.291997 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 00:01:36.291997 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 00:01:36.291997 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 12 00:01:36.291997 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:01:36.291997 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 99/200] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Oct 12 00:01:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 00:01:36.296728 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226260d3-331e-4af5-aff2-ec6875d00e1e None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:36.331400 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:36.424687 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226260d3-331e-4af5-aff2-ec6875d00e1e None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:36.462712 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6dd1b4-acb1-4d79-bcfe-6367551a4be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:36.463332 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6dd1b4-acb1-4d79-bcfe-6367551a4be2 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:36.463647 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6dd1b4-acb1-4d79-bcfe-6367551a4be2 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:36.463919 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6dd1b4-acb1-4d79-bcfe-6367551a4be2 None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:36.464591 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6dd1b4-acb1-4d79-bcfe-6367551a4be2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:01:36.464890 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6dd1b4-acb1-4d79-bcfe-6367551a4be2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:36.466280 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6dd1b4-acb1-4d79-bcfe-6367551a4be2 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:36.475450 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 102/201] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Oct 12 00:01:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:36.481276 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02558340-7a39-4a1b-938b-de16894fd464 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:36.512683 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:36.595150 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02558340-7a39-4a1b-938b-de16894fd464 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:36.632758 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbc279c-ab8c-4d8d-8ba6-2a18b43f4d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:36.633402 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbc279c-ab8c-4d8d-8ba6-2a18b43f4d82 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:36.633683 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbc279c-ab8c-4d8d-8ba6-2a18b43f4d82 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:36.633957 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbc279c-ab8c-4d8d-8ba6-2a18b43f4d82 None admin] PATH_INFO: `/v3/users/4ef91a6924f74989bb81ed49abf61a4d` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:36.635400 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbc279c-ab8c-4d8d-8ba6-2a18b43f4d82 None admin] RBAC: Authorizing `identity:get_user(user_id=4ef91a6924f74989bb81ed49abf61a4d)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:36.636858 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbc279c-ab8c-4d8d-8ba6-2a18b43f4d82 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:36.638887 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 100/202] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Oct 12 00:01:36 2022] GET /identity/v3/users/4ef91a6924f74989bb81ed49abf61a4d => generated 297 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:36.643937 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6dd1b4-acb1-4d79-bcfe-6367551a4be2 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:36.735936 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:36.752335 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6dd1b4-acb1-4d79-bcfe-6367551a4be2 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:36.788289 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0465775d-935a-4e7c-91d3-1b6623e3d1d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:36.788904 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0465775d-935a-4e7c-91d3-1b6623e3d1d2 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:36.789180 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0465775d-935a-4e7c-91d3-1b6623e3d1d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:36.789449 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0465775d-935a-4e7c-91d3-1b6623e3d1d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:36.860511 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0465775d-935a-4e7c-91d3-1b6623e3d1d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:36.862005 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0465775d-935a-4e7c-91d3-1b6623e3d1d2 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:36.947804 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:36.963829 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 103/203] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Oct 12 00:01:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 12 00:01:36.968992 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbc279c-ab8c-4d8d-8ba6-2a18b43f4d82 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:37.068916 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:37.079198 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbc279c-ab8c-4d8d-8ba6-2a18b43f4d82 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:37.121876 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d2814b-a43f-46e1-b4e4-44ecd01407a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:37.121876 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d2814b-a43f-46e1-b4e4-44ecd01407a1 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:37.121876 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d2814b-a43f-46e1-b4e4-44ecd01407a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:37.121876 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d2814b-a43f-46e1-b4e4-44ecd01407a1 None admin] PATH_INFO: `/v3/projects/25ba77a2f84b41cd99a774b93ada5e96` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:37.122273 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d2814b-a43f-46e1-b4e4-44ecd01407a1 None admin] RBAC: Authorizing `identity:get_project(project_id=25ba77a2f84b41cd99a774b93ada5e96)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:37.122273 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d2814b-a43f-46e1-b4e4-44ecd01407a1 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:37.122273 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 101/204] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Oct 12 00:01:36 2022] GET /identity/v3/projects/25ba77a2f84b41cd99a774b93ada5e96 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:37.127472 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0465775d-935a-4e7c-91d3-1b6623e3d1d2 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:37.232594 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0465775d-935a-4e7c-91d3-1b6623e3d1d2 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:37.268168 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b028aa5d-3943-4589-a063-80194eac3f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:37.269149 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b028aa5d-3943-4589-a063-80194eac3f1a None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:37.269626 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b028aa5d-3943-4589-a063-80194eac3f1a None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:37.270137 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b028aa5d-3943-4589-a063-80194eac3f1a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:37.271079 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b028aa5d-3943-4589-a063-80194eac3f1a None admin] RBAC: Adding query filter params (role.id=ef689f96442945e8a7bc8cbfee13d909, scope.project.id=25ba77a2f84b41cd99a774b93ada5e96, user.id=4ef91a6924f74989bb81ed49abf61a4d) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:01:37.271635 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b028aa5d-3943-4589-a063-80194eac3f1a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:37.273266 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b028aa5d-3943-4589-a063-80194eac3f1a None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:37.279428 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 104/205] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Oct 12 00:01:37 2022] GET /identity/v3/role_assignments?user.id=4ef91a6924f74989bb81ed49abf61a4d&scope.project.id=25ba77a2f84b41cd99a774b93ada5e96&role.id=ef689f96442945e8a7bc8cbfee13d909 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:38.128756 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec00596-68de-44e8-91ad-fb463593a326 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:38.129299 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec00596-68de-44e8-91ad-fb463593a326 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:38.129299 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec00596-68de-44e8-91ad-fb463593a326 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:38.130826 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 102/206] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:01:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:01:38.136377 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb98995-ed49-4aa6-be83-6adc309558f6 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:38.136758 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb98995-ed49-4aa6-be83-6adc309558f6 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:38.136928 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb98995-ed49-4aa6-be83-6adc309558f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:38.155519 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-0bb98995-ed49-4aa6-be83-6adc309558f6 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:01:38.156722 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-0bb98995-ed49-4aa6-be83-6adc309558f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:01:38.201693 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5623f-253e-43e2-8a85-db3e2a976f49 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:38.201973 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5623f-253e-43e2-8a85-db3e2a976f49 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:38.202447 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5623f-253e-43e2-8a85-db3e2a976f49 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:38.204374 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:38.204607 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 103/207] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:01:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:01:38.209357 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bf62fa-039a-4868-9dc4-78592e58bb1e None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:38.209636 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bf62fa-039a-4868-9dc4-78592e58bb1e None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:38.218861 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bf62fa-039a-4868-9dc4-78592e58bb1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:38.228313 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 105/208] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:01:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 00:01:38.228533 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-97bf62fa-039a-4868-9dc4-78592e58bb1e None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:01:38.229728 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-97bf62fa-039a-4868-9dc4-78592e58bb1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:01:38.300553 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 104/209] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:01:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 12 00:01:38.418762 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222c682d-00bc-401f-8cc9-0987a3ce2455 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:38.418946 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222c682d-00bc-401f-8cc9-0987a3ce2455 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:38.419217 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222c682d-00bc-401f-8cc9-0987a3ce2455 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:38.420750 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 106/210] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:01:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:01:38.425660 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b889681-712b-4d2d-84e1-c495ce1ba193 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:38.425946 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b889681-712b-4d2d-84e1-c495ce1ba193 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:38.426219 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b889681-712b-4d2d-84e1-c495ce1ba193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:38.444344 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-0b889681-712b-4d2d-84e1-c495ce1ba193 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:01:38.445508 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-0b889681-712b-4d2d-84e1-c495ce1ba193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:01:38.504212 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222c682d-00bc-401f-8cc9-0987a3ce2455 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:38.513209 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 105/211] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:01:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 00:01:38.569230 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b889681-712b-4d2d-84e1-c495ce1ba193 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:38.595038 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:38.619670 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222c682d-00bc-401f-8cc9-0987a3ce2455 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:38.654815 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4775f88f-6347-4397-94c4-e78d10585185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:38.655630 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4775f88f-6347-4397-94c4-e78d10585185 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:38.656223 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4775f88f-6347-4397-94c4-e78d10585185 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:38.656700 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4775f88f-6347-4397-94c4-e78d10585185 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:38.667859 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4775f88f-6347-4397-94c4-e78d10585185 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 00:01:38.674625 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-4775f88f-6347-4397-94c4-e78d10585185 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 12 00:01:38.674625 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:01:38.674625 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:01:38.674625 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:01:38.674625 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:01:38.674625 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:01:38.674625 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:01:38.674625 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:01:38.674625 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:01:38.674625 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:01:38.674625 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:01:38.674625 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:01:38.674625 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 00:01:38.674625 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 00:01:38.674625 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 00:01:38.674625 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 00:01:38.675811 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 00:01:38.675811 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 00:01:38.675811 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 00:01:38.675811 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 00:01:38.675811 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:01:38.675811 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:01:38.675811 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:01:38.675811 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:01:38.675811 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:01:38.675811 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:01:38.675811 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:01:38.675811 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:01:38.675811 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:01:38.675811 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:01:38.675811 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:01:38.675811 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:01:38.677051 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:01:38.677051 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:01:38.677051 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:01:38.677051 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:01:38.677051 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 00:01:38.677051 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 00:01:38.677051 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 00:01:38.677051 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 00:01:38.677051 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 00:01:38.677051 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 00:01:38.677051 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 12 00:01:38.677051 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:01:38.677051 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 107/212] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Oct 12 00:01:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:01:38.681424 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4775f88f-6347-4397-94c4-e78d10585185 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:38.682783 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b889681-712b-4d2d-84e1-c495ce1ba193 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:38.718969 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b20786-07b4-4f8d-867c-41a19976cc3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:38.719631 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b20786-07b4-4f8d-867c-41a19976cc3e None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:38.719909 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b20786-07b4-4f8d-867c-41a19976cc3e None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:38.720177 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b20786-07b4-4f8d-867c-41a19976cc3e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:38.721276 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b20786-07b4-4f8d-867c-41a19976cc3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:38.721720 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:38.723134 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b20786-07b4-4f8d-867c-41a19976cc3e None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:38.725344 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 106/213] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:01:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 00:01:38.730416 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b20786-07b4-4f8d-867c-41a19976cc3e None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:38.784270 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4775f88f-6347-4397-94c4-e78d10585185 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:38.817400 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698d2a91-9756-43fc-91de-efc6f823a8fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:38.818525 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698d2a91-9756-43fc-91de-efc6f823a8fe None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:38.818858 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:38.819191 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698d2a91-9756-43fc-91de-efc6f823a8fe None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:38.819496 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698d2a91-9756-43fc-91de-efc6f823a8fe None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:38.820163 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698d2a91-9756-43fc-91de-efc6f823a8fe None admin] RBAC: Adding query filter params (name=member) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:01:38.820454 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698d2a91-9756-43fc-91de-efc6f823a8fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:38.821824 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698d2a91-9756-43fc-91de-efc6f823a8fe None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:38.829570 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 108/214] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Oct 12 00:01:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 00:01:38.830907 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698d2a91-9756-43fc-91de-efc6f823a8fe None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:38.832047 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b20786-07b4-4f8d-867c-41a19976cc3e None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:38.868882 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7232f372-a229-470e-a5cd-3b7e68faf020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:38.869494 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7232f372-a229-470e-a5cd-3b7e68faf020 None admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:38.869768 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7232f372-a229-470e-a5cd-3b7e68faf020 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:38.870038 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7232f372-a229-470e-a5cd-3b7e68faf020 None admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:38.870679 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7232f372-a229-470e-a5cd-3b7e68faf020 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:38.872137 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7232f372-a229-470e-a5cd-3b7e68faf020 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:38.924261 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:38.927153 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698d2a91-9756-43fc-91de-efc6f823a8fe None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:38.960340 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cf09c2-d528-4f01-ba0f-9ecf21ff8132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:38.960971 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cf09c2-d528-4f01-ba0f-9ecf21ff8132 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:38.961250 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cf09c2-d528-4f01-ba0f-9ecf21ff8132 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:38.961519 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cf09c2-d528-4f01-ba0f-9ecf21ff8132 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:38.968511 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 107/215] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Oct 12 00:01:38 2022] POST /identity/v3/users => generated 293 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 00:01:38.968511 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7232f372-a229-470e-a5cd-3b7e68faf020 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:38.979732 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cf09c2-d528-4f01-ba0f-9ecf21ff8132 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 00:01:38.979732 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-78cf09c2-d528-4f01-ba0f-9ecf21ff8132 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 12 00:01:38.979732 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:01:38.979732 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:01:38.979732 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:01:38.979732 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:01:38.979732 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:01:38.979732 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:01:38.979732 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:01:38.979732 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:01:38.979732 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:01:38.979732 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:01:38.979732 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:01:38.979732 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 00:01:38.980603 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 00:01:38.980603 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 00:01:38.980603 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 00:01:38.980603 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 00:01:38.980603 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 00:01:38.980603 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 00:01:38.980603 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 00:01:38.980603 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:01:38.980603 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:01:38.980603 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:01:38.980603 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:01:38.980603 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:01:38.980603 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:01:38.980603 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:01:38.980603 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:01:38.980603 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:01:38.980603 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:01:38.981616 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:01:38.981616 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:01:38.981616 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:01:38.981616 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:01:38.981616 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:01:38.981616 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:01:38.981616 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 00:01:38.981616 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 00:01:38.981616 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 00:01:38.981616 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 00:01:38.981616 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 00:01:38.981616 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 00:01:38.981616 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 12 00:01:38.981616 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:01:38.981616 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 109/216] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Oct 12 00:01:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:01:38.991433 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cf09c2-d528-4f01-ba0f-9ecf21ff8132 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:39.087162 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7232f372-a229-470e-a5cd-3b7e68faf020 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:39.108024 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cf09c2-d528-4f01-ba0f-9ecf21ff8132 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:39.127223 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ffbc34-971a-4051-b418-a6c3daa5e205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:39.127892 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ffbc34-971a-4051-b418-a6c3daa5e205 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:39.128162 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ffbc34-971a-4051-b418-a6c3daa5e205 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:39.128425 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ffbc34-971a-4051-b418-a6c3daa5e205 None admin] PATH_INFO: `/v3/users/d5691df7a7374d42a6531e4ffbb153b6` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:39.143916 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ffbc34-971a-4051-b418-a6c3daa5e205 None admin] RBAC: Authorizing `identity:get_user(user_id=d5691df7a7374d42a6531e4ffbb153b6)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:39.145376 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ffbc34-971a-4051-b418-a6c3daa5e205 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:39.145503 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29755037-fee3-44b0-8fb8-09276d38c304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:39.146562 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29755037-fee3-44b0-8fb8-09276d38c304 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:39.146807 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29755037-fee3-44b0-8fb8-09276d38c304 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:39.147076 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29755037-fee3-44b0-8fb8-09276d38c304 None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:39.148061 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 108/217] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Oct 12 00:01:38 2022] GET /identity/v3/users/d5691df7a7374d42a6531e4ffbb153b6 => generated 293 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:39.148181 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29755037-fee3-44b0-8fb8-09276d38c304 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:01:39.148181 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29755037-fee3-44b0-8fb8-09276d38c304 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:39.149452 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29755037-fee3-44b0-8fb8-09276d38c304 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:39.153793 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ffbc34-971a-4051-b418-a6c3daa5e205 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:39.157804 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:39.165023 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 110/218] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Oct 12 00:01:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 00:01:39.166816 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29755037-fee3-44b0-8fb8-09276d38c304 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:39.195723 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:39.276791 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29755037-fee3-44b0-8fb8-09276d38c304 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:39.286807 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ffbc34-971a-4051-b418-a6c3daa5e205 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:39.320693 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9138661-b429-4ac4-8f6a-3535c49a6ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:39.321351 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9138661-b429-4ac4-8f6a-3535c49a6ddf None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:39.321650 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9138661-b429-4ac4-8f6a-3535c49a6ddf None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:39.321905 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9138661-b429-4ac4-8f6a-3535c49a6ddf None admin] PATH_INFO: `/v3/users/d92470f780ec4fc3becaebe7ea052bce` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:39.323065 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9138661-b429-4ac4-8f6a-3535c49a6ddf None admin] RBAC: Authorizing `identity:get_user(user_id=d92470f780ec4fc3becaebe7ea052bce)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:39.324490 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9138661-b429-4ac4-8f6a-3535c49a6ddf None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:39.326291 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 111/219] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Oct 12 00:01:39 2022] GET /identity/v3/users/d92470f780ec4fc3becaebe7ea052bce => generated 297 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 00:01:39.327551 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab178019-fa5a-4f52-ba17-af9edcf15ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:39.328170 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab178019-fa5a-4f52-ba17-af9edcf15ca1 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:39.328450 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab178019-fa5a-4f52-ba17-af9edcf15ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:39.328724 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab178019-fa5a-4f52-ba17-af9edcf15ca1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:39.329458 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab178019-fa5a-4f52-ba17-af9edcf15ca1 None admin] RBAC: Adding query filter params (role.id=ef689f96442945e8a7bc8cbfee13d909, user.id=d5691df7a7374d42a6531e4ffbb153b6, scope.system=all) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:01:39.329766 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab178019-fa5a-4f52-ba17-af9edcf15ca1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:39.331305 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab178019-fa5a-4f52-ba17-af9edcf15ca1 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:39.331501 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9138661-b429-4ac4-8f6a-3535c49a6ddf None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:39.338730 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 109/220] 10.222.0.36 () {56 vars in 1308 bytes} [Wed Oct 12 00:01:39 2022] GET /identity/v3/role_assignments?user.id=d5691df7a7374d42a6531e4ffbb153b6&scope.system=all&role.id=ef689f96442945e8a7bc8cbfee13d909 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 00:01:39.439933 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:39.454053 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9138661-b429-4ac4-8f6a-3535c49a6ddf None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:39.489726 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba747e54-04e2-4353-8db2-fa371b580edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:39.490360 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba747e54-04e2-4353-8db2-fa371b580edb None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:39.490634 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba747e54-04e2-4353-8db2-fa371b580edb None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:39.490901 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba747e54-04e2-4353-8db2-fa371b580edb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:39.563599 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba747e54-04e2-4353-8db2-fa371b580edb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:39.565077 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba747e54-04e2-4353-8db2-fa371b580edb None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:39.657584 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:39.668214 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 112/221] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Oct 12 00:01:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 12 00:01:39.677106 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab178019-fa5a-4f52-ba17-af9edcf15ca1 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:39.743184 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:39.778383 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab178019-fa5a-4f52-ba17-af9edcf15ca1 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:39.812643 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25601b9-4e3b-48db-ad0a-9c2709cfc04f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:39.813275 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25601b9-4e3b-48db-ad0a-9c2709cfc04f None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:39.813545 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25601b9-4e3b-48db-ad0a-9c2709cfc04f None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:39.813812 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25601b9-4e3b-48db-ad0a-9c2709cfc04f None admin] PATH_INFO: `/v3/projects/25ba77a2f84b41cd99a774b93ada5e96` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:39.814941 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25601b9-4e3b-48db-ad0a-9c2709cfc04f None admin] RBAC: Authorizing `identity:get_project(project_id=25ba77a2f84b41cd99a774b93ada5e96)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:39.816398 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25601b9-4e3b-48db-ad0a-9c2709cfc04f None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:39.818934 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 110/222] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Oct 12 00:01:39 2022] GET /identity/v3/projects/25ba77a2f84b41cd99a774b93ada5e96 => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:39.824820 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba747e54-04e2-4353-8db2-fa371b580edb None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:39.930355 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba747e54-04e2-4353-8db2-fa371b580edb None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:39.965945 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48945147-c3c4-4715-b49d-ed0dc7327ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:39.966554 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48945147-c3c4-4715-b49d-ed0dc7327ed6 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:39.966826 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48945147-c3c4-4715-b49d-ed0dc7327ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:39.967091 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48945147-c3c4-4715-b49d-ed0dc7327ed6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:39.967846 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48945147-c3c4-4715-b49d-ed0dc7327ed6 None admin] RBAC: Adding query filter params (role.id=217d497bae80496a9d3772fbb90678e1, scope.project.id=25ba77a2f84b41cd99a774b93ada5e96, user.id=d92470f780ec4fc3becaebe7ea052bce) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:01:39.968152 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48945147-c3c4-4715-b49d-ed0dc7327ed6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:39.969658 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48945147-c3c4-4715-b49d-ed0dc7327ed6 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:39.976291 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 113/223] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Oct 12 00:01:39 2022] GET /identity/v3/role_assignments?user.id=d92470f780ec4fc3becaebe7ea052bce&scope.project.id=25ba77a2f84b41cd99a774b93ada5e96&role.id=217d497bae80496a9d3772fbb90678e1 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:41.272444 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e97e64e-460c-418e-a831-4242fcb0ae7d None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:41.272980 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e97e64e-460c-418e-a831-4242fcb0ae7d None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:41.272980 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e97e64e-460c-418e-a831-4242fcb0ae7d None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:41.274394 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 111/224] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:01:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:01:41.280743 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0d030c-8e9c-4a52-aff3-b4735c744413 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:41.281279 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0d030c-8e9c-4a52-aff3-b4735c744413 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:41.281505 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0d030c-8e9c-4a52-aff3-b4735c744413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:41.301160 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-ee0d030c-8e9c-4a52-aff3-b4735c744413 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:01:41.302433 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-ee0d030c-8e9c-4a52-aff3-b4735c744413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:01:41.352517 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:41.370089 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 114/225] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:01:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 00:01:41.395473 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba78b9c-740c-41bd-82e8-5767c3041b82 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:41.396304 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba78b9c-740c-41bd-82e8-5767c3041b82 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:41.396586 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba78b9c-740c-41bd-82e8-5767c3041b82 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:41.398103 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 112/226] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:01:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:01:41.403731 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45021e93-b11b-4459-a8b2-d2827d0c4f8b None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:41.404046 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45021e93-b11b-4459-a8b2-d2827d0c4f8b None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:41.404281 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45021e93-b11b-4459-a8b2-d2827d0c4f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:41.423977 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-45021e93-b11b-4459-a8b2-d2827d0c4f8b None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:01:41.425193 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-45021e93-b11b-4459-a8b2-d2827d0c4f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:01:41.492793 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 115/227] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:01:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 00:01:41.523132 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f0cd61-9af1-4ba0-9a05-ebed75014c68 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:41.523355 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f0cd61-9af1-4ba0-9a05-ebed75014c68 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:41.523762 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f0cd61-9af1-4ba0-9a05-ebed75014c68 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:41.525349 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 113/228] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:01:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:01:41.530315 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c90d3f-a965-4455-b489-c158ccb9ca0f None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:41.530605 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c90d3f-a965-4455-b489-c158ccb9ca0f None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:41.530879 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c90d3f-a965-4455-b489-c158ccb9ca0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:41.555417 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-26c90d3f-a965-4455-b489-c158ccb9ca0f None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:01:41.556616 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-26c90d3f-a965-4455-b489-c158ccb9ca0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:01:41.623406 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 116/229] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:01:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 00:01:41.639134 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f0cd61-9af1-4ba0-9a05-ebed75014c68 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:41.739500 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f0cd61-9af1-4ba0-9a05-ebed75014c68 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:41.755778 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c90d3f-a965-4455-b489-c158ccb9ca0f None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:41.777259 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7967c1-def7-4beb-8baa-e853b1f133ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:41.777823 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7967c1-def7-4beb-8baa-e853b1f133ad None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:41.778100 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7967c1-def7-4beb-8baa-e853b1f133ad None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:41.778371 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7967c1-def7-4beb-8baa-e853b1f133ad None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:41.780364 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:41.790333 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7967c1-def7-4beb-8baa-e853b1f133ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 00:01:41.805560 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-cc7967c1-def7-4beb-8baa-e853b1f133ad None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 12 00:01:41.805560 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:01:41.805560 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:01:41.805560 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:01:41.805560 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:01:41.805560 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:01:41.805560 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:01:41.805560 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:01:41.805560 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:01:41.805560 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:01:41.805560 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:01:41.805560 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:01:41.805560 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 00:01:41.805560 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 00:01:41.805560 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 00:01:41.805560 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 00:01:41.806673 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 00:01:41.806673 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 00:01:41.806673 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 00:01:41.806673 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 00:01:41.806673 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:01:41.806673 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:01:41.806673 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:01:41.806673 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:01:41.806673 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:01:41.806673 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:01:41.806673 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:01:41.806673 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:01:41.806673 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:01:41.806673 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:01:41.806673 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:01:41.806673 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:01:41.807707 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:01:41.807707 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:01:41.807707 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:01:41.807707 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:01:41.807707 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 00:01:41.807707 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 00:01:41.807707 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 00:01:41.807707 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 00:01:41.807707 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 00:01:41.807707 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 00:01:41.807707 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 12 00:01:41.807707 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:01:41.807707 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 114/230] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Oct 12 00:01:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:01:41.807707 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7967c1-def7-4beb-8baa-e853b1f133ad None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:41.808578 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:41.862013 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c90d3f-a965-4455-b489-c158ccb9ca0f None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:41.896476 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc75931-8d74-409c-87c8-3038b7a47d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:41.897104 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc75931-8d74-409c-87c8-3038b7a47d7f None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:41.897378 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc75931-8d74-409c-87c8-3038b7a47d7f None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:41.897645 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc75931-8d74-409c-87c8-3038b7a47d7f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:41.898732 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc75931-8d74-409c-87c8-3038b7a47d7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:41.900226 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc75931-8d74-409c-87c8-3038b7a47d7f None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:41.901781 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 117/231] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:01:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 00:01:41.907049 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc75931-8d74-409c-87c8-3038b7a47d7f None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:41.920443 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7967c1-def7-4beb-8baa-e853b1f133ad None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:41.953946 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb4bf65-8958-4116-96ec-8ca664e7f363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:41.954547 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb4bf65-8958-4116-96ec-8ca664e7f363 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:41.954819 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb4bf65-8958-4116-96ec-8ca664e7f363 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:41.955094 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb4bf65-8958-4116-96ec-8ca664e7f363 None admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:41.955777 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb4bf65-8958-4116-96ec-8ca664e7f363 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:01:41.956071 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb4bf65-8958-4116-96ec-8ca664e7f363 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:41.957440 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb4bf65-8958-4116-96ec-8ca664e7f363 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:41.965165 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 115/232] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Oct 12 00:01:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 00:01:41.968207 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb4bf65-8958-4116-96ec-8ca664e7f363 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:42.004550 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc75931-8d74-409c-87c8-3038b7a47d7f None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:42.017520 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:42.023616 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:42.037691 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153f2f0c-1ec2-4312-b27d-274539da8f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:42.038269 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f2f0c-1ec2-4312-b27d-274539da8f7d None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:42.038556 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f2f0c-1ec2-4312-b27d-274539da8f7d None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:42.038826 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f2f0c-1ec2-4312-b27d-274539da8f7d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:42.048915 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153f2f0c-1ec2-4312-b27d-274539da8f7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 00:01:42.055303 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-153f2f0c-1ec2-4312-b27d-274539da8f7d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 12 00:01:42.055303 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:01:42.055303 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:01:42.055303 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:01:42.055303 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:01:42.055303 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:01:42.055303 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:01:42.055303 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:01:42.055303 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:01:42.055303 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:01:42.055303 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:01:42.055303 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:01:42.055303 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 00:01:42.055303 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 00:01:42.055303 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 00:01:42.055303 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 00:01:42.059115 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 00:01:42.059115 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 00:01:42.059115 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 00:01:42.059115 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 00:01:42.059115 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:01:42.059115 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:01:42.059115 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:01:42.059115 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:01:42.059115 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:01:42.059115 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:01:42.059115 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:01:42.059115 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:01:42.059115 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:01:42.059115 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:01:42.059115 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:01:42.059115 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:01:42.060150 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:01:42.060150 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:01:42.060150 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:01:42.060150 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:01:42.060150 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 00:01:42.060150 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 00:01:42.060150 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 00:01:42.060150 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 00:01:42.060150 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 00:01:42.060150 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 00:01:42.060150 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 12 00:01:42.060150 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:01:42.060150 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 118/233] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Oct 12 00:01:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:01:42.060150 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153f2f0c-1ec2-4312-b27d-274539da8f7d None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:42.068804 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb4bf65-8958-4116-96ec-8ca664e7f363 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:42.101867 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f343649d-d1c1-4674-b9c2-2247ced7ca2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:42.102497 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f343649d-d1c1-4674-b9c2-2247ced7ca2e None admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:42.102785 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f343649d-d1c1-4674-b9c2-2247ced7ca2e None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:42.103080 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f343649d-d1c1-4674-b9c2-2247ced7ca2e None admin] PATH_INFO: `/v3/groups` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:42.103734 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f343649d-d1c1-4674-b9c2-2247ced7ca2e None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:42.105158 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f343649d-d1c1-4674-b9c2-2247ced7ca2e None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:42.156489 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153f2f0c-1ec2-4312-b27d-274539da8f7d None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:42.179830 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 116/234] 10.222.0.36 () {60 vars in 1134 bytes} [Wed Oct 12 00:01:41 2022] POST /identity/v3/groups => generated 229 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:01:42.180695 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f343649d-d1c1-4674-b9c2-2247ced7ca2e None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:42.199297 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c014ddda-4623-429f-8fed-047f2930a812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:42.200358 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c014ddda-4623-429f-8fed-047f2930a812 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:42.200627 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c014ddda-4623-429f-8fed-047f2930a812 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:42.200913 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c014ddda-4623-429f-8fed-047f2930a812 None admin] PATH_INFO: `/v3/users/8d1992b609284eb5b9d51c1a17b4c4d4` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:42.216346 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c014ddda-4623-429f-8fed-047f2930a812 None admin] RBAC: Authorizing `identity:get_user(user_id=8d1992b609284eb5b9d51c1a17b4c4d4)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:42.217875 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c014ddda-4623-429f-8fed-047f2930a812 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:42.219856 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 119/235] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Oct 12 00:01:42 2022] GET /identity/v3/users/8d1992b609284eb5b9d51c1a17b4c4d4 => generated 293 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:42.226066 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c014ddda-4623-429f-8fed-047f2930a812 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:42.259153 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:42.297065 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f343649d-d1c1-4674-b9c2-2247ced7ca2e None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:42.316964 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:42.336831 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ea1bf1-2311-41a9-ae5a-fa6676af7bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:42.337462 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea1bf1-2311-41a9-ae5a-fa6676af7bf6 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:42.337898 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea1bf1-2311-41a9-ae5a-fa6676af7bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:42.338173 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea1bf1-2311-41a9-ae5a-fa6676af7bf6 None admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:42.339490 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ea1bf1-2311-41a9-ae5a-fa6676af7bf6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:01:42.340208 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ea1bf1-2311-41a9-ae5a-fa6676af7bf6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:42.341626 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c014ddda-4623-429f-8fed-047f2930a812 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:42.342345 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ea1bf1-2311-41a9-ae5a-fa6676af7bf6 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:42.353098 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 117/236] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Oct 12 00:01:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:42.358378 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ea1bf1-2311-41a9-ae5a-fa6676af7bf6 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:42.419131 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c951b7-2e50-48e4-80de-907996b4d958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:42.419795 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c951b7-2e50-48e4-80de-907996b4d958 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:42.420069 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c951b7-2e50-48e4-80de-907996b4d958 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:42.420332 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c951b7-2e50-48e4-80de-907996b4d958 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:42.421092 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c951b7-2e50-48e4-80de-907996b4d958 None admin] RBAC: Adding query filter params (role.id=217d497bae80496a9d3772fbb90678e1, user.id=8d1992b609284eb5b9d51c1a17b4c4d4, scope.system=all) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:01:42.421392 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c951b7-2e50-48e4-80de-907996b4d958 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:42.422824 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c951b7-2e50-48e4-80de-907996b4d958 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:42.428723 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 120/237] 10.222.0.36 () {56 vars in 1308 bytes} [Wed Oct 12 00:01:42 2022] GET /identity/v3/role_assignments?user.id=8d1992b609284eb5b9d51c1a17b4c4d4&scope.system=all&role.id=217d497bae80496a9d3772fbb90678e1 => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 00:01:42.474996 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ea1bf1-2311-41a9-ae5a-fa6676af7bf6 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:42.512965 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe19f5ac-f916-47a6-aa98-09635e69fbac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:42.513616 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe19f5ac-f916-47a6-aa98-09635e69fbac None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:42.513888 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe19f5ac-f916-47a6-aa98-09635e69fbac None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:42.514153 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe19f5ac-f916-47a6-aa98-09635e69fbac None admin] PATH_INFO: `/v3/users/d5691df7a7374d42a6531e4ffbb153b6` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:42.515334 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe19f5ac-f916-47a6-aa98-09635e69fbac None admin] RBAC: Authorizing `identity:get_user(user_id=d5691df7a7374d42a6531e4ffbb153b6)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:42.516846 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe19f5ac-f916-47a6-aa98-09635e69fbac None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:42.518700 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 118/238] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Oct 12 00:01:42 2022] GET /identity/v3/users/d5691df7a7374d42a6531e4ffbb153b6 => generated 293 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 00:01:42.524404 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c951b7-2e50-48e4-80de-907996b4d958 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:42.626944 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:42.634045 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c951b7-2e50-48e4-80de-907996b4d958 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:42.667962 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d6e525-2236-4167-8043-979adfc6f3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:42.668621 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d6e525-2236-4167-8043-979adfc6f3ca None admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:42.668899 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d6e525-2236-4167-8043-979adfc6f3ca None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:42.669173 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d6e525-2236-4167-8043-979adfc6f3ca None admin] PATH_INFO: `/v3/system/users/d5691df7a7374d42a6531e4ffbb153b6/roles/ef689f96442945e8a7bc8cbfee13d909` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:42.670596 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d6e525-2236-4167-8043-979adfc6f3ca None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=d5691df7a7374d42a6531e4ffbb153b6, role_id=ef689f96442945e8a7bc8cbfee13d909)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:42.672015 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d6e525-2236-4167-8043-979adfc6f3ca None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:42.777421 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 121/239] 10.222.0.36 () {58 vars in 1334 bytes} [Wed Oct 12 00:01:42 2022] PUT /identity/v3/system/users/d5691df7a7374d42a6531e4ffbb153b6/roles/ef689f96442945e8a7bc8cbfee13d909 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:01:44.356023 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd34d1f2-6b60-40b5-bd5c-0acbbef731d6 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:44.356543 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd34d1f2-6b60-40b5-bd5c-0acbbef731d6 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:44.356543 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd34d1f2-6b60-40b5-bd5c-0acbbef731d6 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:44.357936 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 119/240] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:01:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:01:44.363042 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b81cf1-f858-4853-bcb9-3a9069f1d3d1 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:44.363443 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b81cf1-f858-4853-bcb9-3a9069f1d3d1 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:44.363626 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b81cf1-f858-4853-bcb9-3a9069f1d3d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:44.382522 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-d7b81cf1-f858-4853-bcb9-3a9069f1d3d1 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:01:44.383732 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-d7b81cf1-f858-4853-bcb9-3a9069f1d3d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:01:44.453599 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 122/241] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:01:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 00:01:44.563695 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad16b6e1-c911-438e-8978-4a856c9dff8b None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:44.563982 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad16b6e1-c911-438e-8978-4a856c9dff8b None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:44.564252 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad16b6e1-c911-438e-8978-4a856c9dff8b None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:44.565736 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 120/242] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:01:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:01:44.570704 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de376a9a-7a30-4baa-afdd-f1f8b843931b None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:44.570998 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de376a9a-7a30-4baa-afdd-f1f8b843931b None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:44.571275 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de376a9a-7a30-4baa-afdd-f1f8b843931b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:44.590261 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-de376a9a-7a30-4baa-afdd-f1f8b843931b None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:01:44.591482 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-de376a9a-7a30-4baa-afdd-f1f8b843931b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:01:44.659229 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 123/243] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:01:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 00:01:44.718070 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad16b6e1-c911-438e-8978-4a856c9dff8b None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:44.766416 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:44.825351 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad16b6e1-c911-438e-8978-4a856c9dff8b None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:44.858617 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66425fe-cdcd-4a1f-8530-c2eb11608961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:44.859232 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66425fe-cdcd-4a1f-8530-c2eb11608961 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:44.859537 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66425fe-cdcd-4a1f-8530-c2eb11608961 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:44.859808 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66425fe-cdcd-4a1f-8530-c2eb11608961 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:44.860890 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66425fe-cdcd-4a1f-8530-c2eb11608961 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:44.862335 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66425fe-cdcd-4a1f-8530-c2eb11608961 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:44.864134 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 121/244] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:01:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:44.869403 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de376a9a-7a30-4baa-afdd-f1f8b843931b None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:44.887935 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:44.898449 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075f6516-d2a3-4a06-a771-7e77dd7df9f7 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:44.903672 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075f6516-d2a3-4a06-a771-7e77dd7df9f7 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:44.903672 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075f6516-d2a3-4a06-a771-7e77dd7df9f7 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:44.903672 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 122/245] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:01:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 00:01:44.905290 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d4fdb3-dbec-4ade-949f-4feb2d5509cf None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:44.905571 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d4fdb3-dbec-4ade-949f-4feb2d5509cf None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:44.905839 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d4fdb3-dbec-4ade-949f-4feb2d5509cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:44.924321 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-32d4fdb3-dbec-4ade-949f-4feb2d5509cf None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:01:44.925448 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-32d4fdb3-dbec-4ade-949f-4feb2d5509cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:01:44.968851 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de376a9a-7a30-4baa-afdd-f1f8b843931b None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:44.992119 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 123/246] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:01:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 12 00:01:44.992938 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d4fdb3-dbec-4ade-949f-4feb2d5509cf None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:45.009146 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de6519d-a1c8-4e44-97f1-25159d09ae9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:45.010003 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de6519d-a1c8-4e44-97f1-25159d09ae9f None admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:45.010286 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de6519d-a1c8-4e44-97f1-25159d09ae9f None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:45.010554 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de6519d-a1c8-4e44-97f1-25159d09ae9f None admin] PATH_INFO: `/v3/groups` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:45.011185 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de6519d-a1c8-4e44-97f1-25159d09ae9f None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:45.012688 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de6519d-a1c8-4e44-97f1-25159d09ae9f None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:45.038517 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:45.089359 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d4fdb3-dbec-4ade-949f-4feb2d5509cf None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:45.117028 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 124/247] 10.222.0.36 () {60 vars in 1134 bytes} [Wed Oct 12 00:01:44 2022] POST /identity/v3/groups => generated 226 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:01:45.122563 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f8da0-1e84-4fc7-8bee-94d893340bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:45.123217 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f8da0-1e84-4fc7-8bee-94d893340bb7 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:45.123526 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f8da0-1e84-4fc7-8bee-94d893340bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:45.123793 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f8da0-1e84-4fc7-8bee-94d893340bb7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:45.135716 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598f8da0-1e84-4fc7-8bee-94d893340bb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 00:01:45.140799 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-598f8da0-1e84-4fc7-8bee-94d893340bb7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 12 00:01:45.140799 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:01:45.140799 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:01:45.140799 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:01:45.140799 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:01:45.140799 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:01:45.140799 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:01:45.140799 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:01:45.140799 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:01:45.140799 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:01:45.140799 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:01:45.140799 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:01:45.140799 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 00:01:45.140799 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 00:01:45.140799 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 00:01:45.140799 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 00:01:45.141790 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 00:01:45.141790 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 00:01:45.141790 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 00:01:45.141790 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 00:01:45.141790 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:01:45.141790 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:01:45.141790 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:01:45.141790 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:01:45.141790 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:01:45.141790 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:01:45.141790 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:01:45.141790 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:01:45.141790 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:01:45.141790 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:01:45.141790 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:01:45.141790 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:01:45.143099 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:01:45.143099 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:01:45.143099 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:01:45.143099 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:01:45.143099 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 00:01:45.143099 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 00:01:45.143099 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 00:01:45.143099 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 00:01:45.143099 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 00:01:45.143099 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 00:01:45.143099 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 12 00:01:45.143099 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:01:45.143099 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 124/248] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Oct 12 00:01:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:01:45.148600 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de6519d-a1c8-4e44-97f1-25159d09ae9f None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:45.236812 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:45.260625 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de6519d-a1c8-4e44-97f1-25159d09ae9f None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:45.267025 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f8da0-1e84-4fc7-8bee-94d893340bb7 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:45.305003 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4928727-231a-4194-96cf-b32f2eba0a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:45.305634 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4928727-231a-4194-96cf-b32f2eba0a93 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:45.305914 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4928727-231a-4194-96cf-b32f2eba0a93 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:45.306183 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4928727-231a-4194-96cf-b32f2eba0a93 None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:45.307734 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4928727-231a-4194-96cf-b32f2eba0a93 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:01:45.307734 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4928727-231a-4194-96cf-b32f2eba0a93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:45.308639 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4928727-231a-4194-96cf-b32f2eba0a93 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:45.318529 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 125/249] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Oct 12 00:01:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 00:01:45.324122 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4928727-231a-4194-96cf-b32f2eba0a93 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:45.378603 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f8da0-1e84-4fc7-8bee-94d893340bb7 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:45.383006 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:45.421222 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aa6dc1-4dde-4eeb-8053-6bb457c5dfe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:45.421866 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa6dc1-4dde-4eeb-8053-6bb457c5dfe6 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:45.422452 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa6dc1-4dde-4eeb-8053-6bb457c5dfe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:45.422729 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa6dc1-4dde-4eeb-8053-6bb457c5dfe6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:45.435399 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aa6dc1-4dde-4eeb-8053-6bb457c5dfe6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 00:01:45.440584 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-f0aa6dc1-4dde-4eeb-8053-6bb457c5dfe6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 12 00:01:45.440584 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:01:45.440584 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:01:45.440584 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:01:45.440584 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:01:45.440584 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:01:45.440584 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:01:45.440584 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:01:45.440584 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:01:45.440584 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:01:45.440584 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:01:45.440584 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:01:45.440584 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 00:01:45.440584 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 00:01:45.440584 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 00:01:45.440584 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 00:01:45.441493 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 00:01:45.441493 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 00:01:45.441493 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 00:01:45.441493 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 00:01:45.441493 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:01:45.441493 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:01:45.441493 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:01:45.441493 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:01:45.441493 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:01:45.441493 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:01:45.441493 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:01:45.441493 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:01:45.441493 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:01:45.441493 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:01:45.441493 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:01:45.441493 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:01:45.442269 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:01:45.442269 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:01:45.442269 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:01:45.442269 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:01:45.442269 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 00:01:45.442269 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 00:01:45.442269 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 00:01:45.442269 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 00:01:45.442269 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 00:01:45.442269 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 00:01:45.442269 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 12 00:01:45.442269 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:01:45.442269 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 125/250] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Oct 12 00:01:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 00:01:45.447801 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4928727-231a-4194-96cf-b32f2eba0a93 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:45.453561 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aa6dc1-4dde-4eeb-8053-6bb457c5dfe6 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:45.487300 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2ec869-752f-40f2-8634-b69cc8939b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:45.487915 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2ec869-752f-40f2-8634-b69cc8939b7a None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:45.488195 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2ec869-752f-40f2-8634-b69cc8939b7a None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:45.488469 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2ec869-752f-40f2-8634-b69cc8939b7a None admin] PATH_INFO: `/v3/users/8d1992b609284eb5b9d51c1a17b4c4d4` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:45.489685 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2ec869-752f-40f2-8634-b69cc8939b7a None admin] RBAC: Authorizing `identity:get_user(user_id=8d1992b609284eb5b9d51c1a17b4c4d4)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:45.491156 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2ec869-752f-40f2-8634-b69cc8939b7a None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:45.493227 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 126/251] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Oct 12 00:01:45 2022] GET /identity/v3/users/8d1992b609284eb5b9d51c1a17b4c4d4 => generated 293 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 00:01:45.498729 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2ec869-752f-40f2-8634-b69cc8939b7a None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:45.512203 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:45.585060 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aa6dc1-4dde-4eeb-8053-6bb457c5dfe6 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:45.619806 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2ec869-752f-40f2-8634-b69cc8939b7a None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:45.626342 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637fd31c-3f80-4f9a-b3bf-e9ab76c594ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:45.627243 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637fd31c-3f80-4f9a-b3bf-e9ab76c594ee None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:45.627778 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637fd31c-3f80-4f9a-b3bf-e9ab76c594ee None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:45.628245 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637fd31c-3f80-4f9a-b3bf-e9ab76c594ee None admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:45.629107 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637fd31c-3f80-4f9a-b3bf-e9ab76c594ee None admin] RBAC: Adding query filter params (name=member) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:01:45.629604 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637fd31c-3f80-4f9a-b3bf-e9ab76c594ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:45.631790 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637fd31c-3f80-4f9a-b3bf-e9ab76c594ee None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:45.646166 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 126/252] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Oct 12 00:01:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 00:01:45.652234 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637fd31c-3f80-4f9a-b3bf-e9ab76c594ee None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:45.672437 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3349c8d8-6696-44a0-b515-dc7c34d0cff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:45.673482 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3349c8d8-6696-44a0-b515-dc7c34d0cff1 None admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:45.674003 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3349c8d8-6696-44a0-b515-dc7c34d0cff1 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:45.674543 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3349c8d8-6696-44a0-b515-dc7c34d0cff1 None admin] PATH_INFO: `/v3/system/users/8d1992b609284eb5b9d51c1a17b4c4d4/roles/217d497bae80496a9d3772fbb90678e1` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:45.676377 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3349c8d8-6696-44a0-b515-dc7c34d0cff1 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=8d1992b609284eb5b9d51c1a17b4c4d4, role_id=217d497bae80496a9d3772fbb90678e1)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:45.678722 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3349c8d8-6696-44a0-b515-dc7c34d0cff1 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:45.686693 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:45.737903 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 127/253] 10.222.0.36 () {58 vars in 1334 bytes} [Wed Oct 12 00:01:45 2022] PUT /identity/v3/system/users/8d1992b609284eb5b9d51c1a17b4c4d4/roles/217d497bae80496a9d3772fbb90678e1 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 00:01:45.776200 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637fd31c-3f80-4f9a-b3bf-e9ab76c594ee None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:45.819642 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece860e8-d7a0-4661-9a72-3ce5bf847663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:45.820300 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece860e8-d7a0-4661-9a72-3ce5bf847663 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:45.820578 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece860e8-d7a0-4661-9a72-3ce5bf847663 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:45.820845 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece860e8-d7a0-4661-9a72-3ce5bf847663 None admin] PATH_INFO: `/v3/users/d5691df7a7374d42a6531e4ffbb153b6` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:45.822064 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece860e8-d7a0-4661-9a72-3ce5bf847663 None admin] RBAC: Authorizing `identity:get_user(user_id=d5691df7a7374d42a6531e4ffbb153b6)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:45.823567 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece860e8-d7a0-4661-9a72-3ce5bf847663 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:45.825610 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 127/254] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Oct 12 00:01:45 2022] GET /identity/v3/users/d5691df7a7374d42a6531e4ffbb153b6 => generated 293 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:45.831058 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3349c8d8-6696-44a0-b515-dc7c34d0cff1 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:45.921397 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:45.960220 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3349c8d8-6696-44a0-b515-dc7c34d0cff1 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:46.003125 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877ebfd2-c8b7-4c65-b540-58823cabac54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:46.003704 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877ebfd2-c8b7-4c65-b540-58823cabac54 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:46.003983 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877ebfd2-c8b7-4c65-b540-58823cabac54 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:46.004256 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877ebfd2-c8b7-4c65-b540-58823cabac54 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:46.005044 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877ebfd2-c8b7-4c65-b540-58823cabac54 None admin] RBAC: Adding query filter params (role.id=ef689f96442945e8a7bc8cbfee13d909, user.id=d5691df7a7374d42a6531e4ffbb153b6, scope.system=all) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:01:46.005351 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877ebfd2-c8b7-4c65-b540-58823cabac54 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:46.006815 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877ebfd2-c8b7-4c65-b540-58823cabac54 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:46.023098 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 128/255] 10.222.0.36 () {56 vars in 1308 bytes} [Wed Oct 12 00:01:45 2022] GET /identity/v3/role_assignments?user.id=d5691df7a7374d42a6531e4ffbb153b6&scope.system=all&role.id=ef689f96442945e8a7bc8cbfee13d909 => generated 510 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:47.812741 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50fc04c-af78-44da-a080-6b0010be2993 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:47.813680 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50fc04c-af78-44da-a080-6b0010be2993 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:47.814169 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50fc04c-af78-44da-a080-6b0010be2993 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:47.816012 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 128/256] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:01:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:01:47.829084 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4493df8-95d8-4c31-b0a2-690ee104e49f None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:47.830028 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4493df8-95d8-4c31-b0a2-690ee104e49f None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:47.830496 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4493df8-95d8-4c31-b0a2-690ee104e49f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:47.853977 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-a4493df8-95d8-4c31-b0a2-690ee104e49f None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:01:47.855589 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-a4493df8-95d8-4c31-b0a2-690ee104e49f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:01:47.888628 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc4a71c-1a2d-4c57-9c54-903685260f49 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:47.889245 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc4a71c-1a2d-4c57-9c54-903685260f49 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:47.889776 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc4a71c-1a2d-4c57-9c54-903685260f49 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:47.891634 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 129/257] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:01:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:01:47.897296 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d8bc16-bee3-4afd-80cc-3d8953ab9e9c None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:47.897830 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d8bc16-bee3-4afd-80cc-3d8953ab9e9c None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:47.898320 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d8bc16-bee3-4afd-80cc-3d8953ab9e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:47.936360 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-19d8bc16-bee3-4afd-80cc-3d8953ab9e9c None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:01:47.938006 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-19d8bc16-bee3-4afd-80cc-3d8953ab9e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:01:47.939865 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 129/258] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:01:47 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 12 00:01:47.954279 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 130/259] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:01:47 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 12 00:01:48.367091 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4493df8-95d8-4c31-b0a2-690ee104e49f None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:48.376417 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4493df8-95d8-4c31-b0a2-690ee104e49f None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:48.388043 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92757db-60e3-4be2-9687-041e6bfd2dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:48.388699 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92757db-60e3-4be2-9687-041e6bfd2dfc admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:48.388977 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92757db-60e3-4be2-9687-041e6bfd2dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:48.389247 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92757db-60e3-4be2-9687-041e6bfd2dfc admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:48.400320 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92757db-60e3-4be2-9687-041e6bfd2dfc admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 00:01:48.405214 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-c92757db-60e3-4be2-9687-041e6bfd2dfc admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 12 00:01:48.405214 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:01:48.405214 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:01:48.405214 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:01:48.405214 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:01:48.405214 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:01:48.405214 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:01:48.405214 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:01:48.405214 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:01:48.405214 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:01:48.405214 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:01:48.405214 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:01:48.405214 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 00:01:48.405214 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 00:01:48.405214 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 00:01:48.405214 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 00:01:48.406095 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 00:01:48.406095 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 00:01:48.406095 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 00:01:48.406095 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 00:01:48.406095 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:01:48.406095 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:01:48.406095 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:01:48.406095 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:01:48.406095 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:01:48.406095 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:01:48.406095 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:01:48.406095 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:01:48.406095 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:01:48.406095 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:01:48.406095 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:01:48.406095 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:01:48.406939 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:01:48.406939 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:01:48.406939 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:01:48.406939 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:01:48.406939 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 00:01:48.406939 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 00:01:48.406939 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 00:01:48.406939 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 00:01:48.406939 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 00:01:48.406939 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 00:01:48.406939 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 12 00:01:48.406939 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:01:48.406939 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 130/260] 10.222.0.36 () {56 vars in 1120 bytes} [Wed Oct 12 00:01:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 40 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:01:48.413061 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d8bc16-bee3-4afd-80cc-3d8953ab9e9c None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:48.427647 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d8bc16-bee3-4afd-80cc-3d8953ab9e9c None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:48.435200 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120711ee-acbf-4f39-b035-d12aba6d3046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:48.436117 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120711ee-acbf-4f39-b035-d12aba6d3046 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:48.436593 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120711ee-acbf-4f39-b035-d12aba6d3046 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:48.437082 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120711ee-acbf-4f39-b035-d12aba6d3046 admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:48.437942 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120711ee-acbf-4f39-b035-d12aba6d3046 admin admin] RBAC: Adding query filter params (name=member) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:01:48.438451 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120711ee-acbf-4f39-b035-d12aba6d3046 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:48.440463 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: WARNING py.warnings [None req-120711ee-acbf-4f39-b035-d12aba6d3046 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 12 00:01:48.440463 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: warnings.warn(msg) Oct 12 00:01:48.440463 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: Oct 12 00:01:48.441766 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120711ee-acbf-4f39-b035-d12aba6d3046 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:48.454571 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92757db-60e3-4be2-9687-041e6bfd2dfc admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:48.458440 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 131/261] 10.222.0.36 () {56 vars in 1122 bytes} [Wed Oct 12 00:01:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:48.468891 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92757db-60e3-4be2-9687-041e6bfd2dfc admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:48.471381 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120711ee-acbf-4f39-b035-d12aba6d3046 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:48.480056 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfdbd09-b15e-445d-a1bc-2c3a8976c4e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:48.480706 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfdbd09-b15e-445d-a1bc-2c3a8976c4e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:48.480983 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfdbd09-b15e-445d-a1bc-2c3a8976c4e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:48.481255 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfdbd09-b15e-445d-a1bc-2c3a8976c4e9 admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:48.483808 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120711ee-acbf-4f39-b035-d12aba6d3046 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:48.488807 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2418729b-4a8c-4378-bcca-37a4a51f13cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:48.490354 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2418729b-4a8c-4378-bcca-37a4a51f13cf admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:48.490924 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2418729b-4a8c-4378-bcca-37a4a51f13cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:48.491437 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2418729b-4a8c-4378-bcca-37a4a51f13cf admin admin] PATH_INFO: `/v3/groups/519fd3c5007f452e8c09b26b1c49ccce` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:48.499138 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2418729b-4a8c-4378-bcca-37a4a51f13cf admin admin] RBAC: Authorizing `identity:get_group(group_id=519fd3c5007f452e8c09b26b1c49ccce)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:48.500470 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: WARNING py.warnings [None req-2418729b-4a8c-4378-bcca-37a4a51f13cf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 12 00:01:48.500470 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: warnings.warn(msg) Oct 12 00:01:48.500470 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: Oct 12 00:01:48.502061 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2418729b-4a8c-4378-bcca-37a4a51f13cf admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:48.502631 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfdbd09-b15e-445d-a1bc-2c3a8976c4e9 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 00:01:48.504547 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 132/262] 10.222.0.36 () {56 vars in 1201 bytes} [Wed Oct 12 00:01:48 2022] GET /identity/v3/groups/519fd3c5007f452e8c09b26b1c49ccce => generated 226 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 00:01:48.511426 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-bbfdbd09-b15e-445d-a1bc-2c3a8976c4e9 admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 12 00:01:48.511426 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:01:48.511426 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:01:48.511426 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:01:48.511426 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:01:48.511426 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:01:48.511426 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:01:48.511426 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:01:48.511426 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:01:48.511426 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:01:48.511426 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:01:48.511426 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:01:48.511426 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 00:01:48.511426 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 00:01:48.511426 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 00:01:48.511426 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 00:01:48.514326 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 00:01:48.514326 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 00:01:48.514326 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 00:01:48.514326 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 00:01:48.514326 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:01:48.514326 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:01:48.514326 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:01:48.514326 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:01:48.514326 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:01:48.514326 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:01:48.514326 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:01:48.514326 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:01:48.514326 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:01:48.514326 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:01:48.514326 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:01:48.514326 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:01:48.515115 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:01:48.515115 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:01:48.515115 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:01:48.515115 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:01:48.515115 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 00:01:48.515115 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 00:01:48.515115 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 00:01:48.515115 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 00:01:48.515115 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 00:01:48.515115 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 00:01:48.515115 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 12 00:01:48.515115 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:01:48.515115 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 131/263] 10.222.0.36 () {56 vars in 1135 bytes} [Wed Oct 12 00:01:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 55 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 00:01:48.515776 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2418729b-4a8c-4378-bcca-37a4a51f13cf admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:48.515861 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfdbd09-b15e-445d-a1bc-2c3a8976c4e9 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:48.528278 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2418729b-4a8c-4378-bcca-37a4a51f13cf admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:48.531112 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfdbd09-b15e-445d-a1bc-2c3a8976c4e9 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:48.532832 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8296610-7ed0-4fc0-9a7a-4f449cd8eb21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:48.533473 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8296610-7ed0-4fc0-9a7a-4f449cd8eb21 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:48.533747 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8296610-7ed0-4fc0-9a7a-4f449cd8eb21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:48.534018 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8296610-7ed0-4fc0-9a7a-4f449cd8eb21 admin admin] PATH_INFO: `/v3/projects/410027d6c764410ea5b76db0171903df` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:48.535158 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8296610-7ed0-4fc0-9a7a-4f449cd8eb21 admin admin] RBAC: Authorizing `identity:get_project(project_id=410027d6c764410ea5b76db0171903df)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:48.545572 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8296610-7ed0-4fc0-9a7a-4f449cd8eb21 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:48.547333 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 133/264] 10.222.0.36 () {56 vars in 1207 bytes} [Wed Oct 12 00:01:48 2022] GET /identity/v3/projects/410027d6c764410ea5b76db0171903df => generated 298 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:48.548643 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe1dccd-67ec-44a8-a6bf-591346f0bfcd None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:48.548921 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe1dccd-67ec-44a8-a6bf-591346f0bfcd None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:48.549191 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe1dccd-67ec-44a8-a6bf-591346f0bfcd None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:48.550640 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 134/265] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:01:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:01:48.552897 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e45692f-2b61-4986-bf53-5396cd1d2ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:48.553523 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e45692f-2b61-4986-bf53-5396cd1d2ca6 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:48.553799 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e45692f-2b61-4986-bf53-5396cd1d2ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:48.554067 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e45692f-2b61-4986-bf53-5396cd1d2ca6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:48.554744 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e45692f-2b61-4986-bf53-5396cd1d2ca6 admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:01:48.555389 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e45692f-2b61-4986-bf53-5396cd1d2ca6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:48.556442 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacaba80-afc3-4a5f-8d00-ee2034d86d95 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:48.556724 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacaba80-afc3-4a5f-8d00-ee2034d86d95 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:48.557001 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacaba80-afc3-4a5f-8d00-ee2034d86d95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:48.557204 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: WARNING py.warnings [None req-6e45692f-2b61-4986-bf53-5396cd1d2ca6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 12 00:01:48.557204 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: warnings.warn(msg) Oct 12 00:01:48.557204 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: Oct 12 00:01:48.558112 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e45692f-2b61-4986-bf53-5396cd1d2ca6 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:48.578562 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-eacaba80-afc3-4a5f-8d00-ee2034d86d95 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:01:48.580088 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-eacaba80-afc3-4a5f-8d00-ee2034d86d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:01:48.583724 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 132/266] 10.222.0.36 () {56 vars in 1132 bytes} [Wed Oct 12 00:01:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 00:01:48.584651 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e45692f-2b61-4986-bf53-5396cd1d2ca6 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:48.589540 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 135/267] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:01:48 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 12 00:01:48.590534 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacaba80-afc3-4a5f-8d00-ee2034d86d95 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:48.600781 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e45692f-2b61-4986-bf53-5396cd1d2ca6 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:48.608838 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacaba80-afc3-4a5f-8d00-ee2034d86d95 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:48.612655 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3da205-4ad3-4945-9f49-395e17e87f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:48.612655 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3da205-4ad3-4945-9f49-395e17e87f5c admin admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:48.612655 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3da205-4ad3-4945-9f49-395e17e87f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:48.612933 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3da205-4ad3-4945-9f49-395e17e87f5c admin admin] PATH_INFO: `/v3/projects/410027d6c764410ea5b76db0171903df/groups/519fd3c5007f452e8c09b26b1c49ccce/roles/ef689f96442945e8a7bc8cbfee13d909` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:48.614890 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3da205-4ad3-4945-9f49-395e17e87f5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=410027d6c764410ea5b76db0171903df, group_id=519fd3c5007f452e8c09b26b1c49ccce, role_id=ef689f96442945e8a7bc8cbfee13d909)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:48.615950 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: WARNING py.warnings [None req-5b3da205-4ad3-4945-9f49-395e17e87f5c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 12 00:01:48.615950 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: warnings.warn(msg) Oct 12 00:01:48.615950 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: Oct 12 00:01:48.618088 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3da205-4ad3-4945-9f49-395e17e87f5c admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:48.618908 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f191bc0-84cc-4e6a-8191-c66500e44dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:48.619811 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f191bc0-84cc-4e6a-8191-c66500e44dd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:48.620257 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f191bc0-84cc-4e6a-8191-c66500e44dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:48.620752 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f191bc0-84cc-4e6a-8191-c66500e44dd7 admin admin] PATH_INFO: `/v3/groups/519fd3c5007f452e8c09b26b1c49ccce` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:48.623029 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f191bc0-84cc-4e6a-8191-c66500e44dd7 admin admin] RBAC: Authorizing `identity:get_group(group_id=519fd3c5007f452e8c09b26b1c49ccce)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:48.625935 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f191bc0-84cc-4e6a-8191-c66500e44dd7 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:48.628725 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 136/268] 10.222.0.36 () {56 vars in 1201 bytes} [Wed Oct 12 00:01:48 2022] GET /identity/v3/groups/519fd3c5007f452e8c09b26b1c49ccce => generated 226 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:48.634264 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f191bc0-84cc-4e6a-8191-c66500e44dd7 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:48.649996 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f191bc0-84cc-4e6a-8191-c66500e44dd7 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:48.657138 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805be0a1-5188-49d1-a232-2d09b46cdebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:48.657994 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805be0a1-5188-49d1-a232-2d09b46cdebd admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:48.658519 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805be0a1-5188-49d1-a232-2d09b46cdebd admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:48.658962 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805be0a1-5188-49d1-a232-2d09b46cdebd admin admin] PATH_INFO: `/v3/projects/410027d6c764410ea5b76db0171903df` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:48.660621 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805be0a1-5188-49d1-a232-2d09b46cdebd admin admin] RBAC: Authorizing `identity:get_project(project_id=410027d6c764410ea5b76db0171903df)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:48.663547 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805be0a1-5188-49d1-a232-2d09b46cdebd admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:48.665771 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 137/269] 10.222.0.36 () {56 vars in 1207 bytes} [Wed Oct 12 00:01:48 2022] GET /identity/v3/projects/410027d6c764410ea5b76db0171903df => generated 298 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:48.671459 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805be0a1-5188-49d1-a232-2d09b46cdebd admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:48.684728 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805be0a1-5188-49d1-a232-2d09b46cdebd admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:48.692322 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47524089-edbf-4056-a159-7fef2f1501fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:48.693219 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47524089-edbf-4056-a159-7fef2f1501fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:48.693707 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47524089-edbf-4056-a159-7fef2f1501fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:48.694119 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47524089-edbf-4056-a159-7fef2f1501fc admin admin] PATH_INFO: `/v3/projects/410027d6c764410ea5b76db0171903df/groups/519fd3c5007f452e8c09b26b1c49ccce/roles/81991615ac0b46818426bf37015df930` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:48.697880 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47524089-edbf-4056-a159-7fef2f1501fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=410027d6c764410ea5b76db0171903df, group_id=519fd3c5007f452e8c09b26b1c49ccce, role_id=81991615ac0b46818426bf37015df930)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:48.698237 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 133/270] 10.222.0.36 () {58 vars in 1463 bytes} [Wed Oct 12 00:01:48 2022] PUT /identity/v3/projects/410027d6c764410ea5b76db0171903df/groups/519fd3c5007f452e8c09b26b1c49ccce/roles/ef689f96442945e8a7bc8cbfee13d909 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:01:48.699290 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: WARNING py.warnings [None req-47524089-edbf-4056-a159-7fef2f1501fc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 12 00:01:48.699290 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: warnings.warn(msg) Oct 12 00:01:48.699290 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: Oct 12 00:01:48.701896 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47524089-edbf-4056-a159-7fef2f1501fc admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:48.814831 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 138/271] 10.222.0.36 () {58 vars in 1463 bytes} [Wed Oct 12 00:01:48 2022] PUT /identity/v3/projects/410027d6c764410ea5b76db0171903df/groups/519fd3c5007f452e8c09b26b1c49ccce/roles/81991615ac0b46818426bf37015df930 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 00:01:49.005298 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f048758-fcf9-4f3f-b1c8-bdcc1f6c3d06 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:49.005802 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f048758-fcf9-4f3f-b1c8-bdcc1f6c3d06 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:49.005802 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f048758-fcf9-4f3f-b1c8-bdcc1f6c3d06 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:49.007192 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 134/272] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:01:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:01:49.013373 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177392dc-e7bd-481c-b779-dabb82f22a2d None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:49.013977 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177392dc-e7bd-481c-b779-dabb82f22a2d None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:49.014443 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177392dc-e7bd-481c-b779-dabb82f22a2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:49.037405 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-177392dc-e7bd-481c-b779-dabb82f22a2d None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:01:49.039047 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-177392dc-e7bd-481c-b779-dabb82f22a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:01:49.080055 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:49.092198 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f048758-fcf9-4f3f-b1c8-bdcc1f6c3d06 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:49.127641 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 139/273] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:01:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 00:01:49.145641 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:49.177710 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f048758-fcf9-4f3f-b1c8-bdcc1f6c3d06 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:49.183823 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ed5f07-8d95-4ace-b809-33dedeff226a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:49.184470 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ed5f07-8d95-4ace-b809-33dedeff226a admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:49.184745 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ed5f07-8d95-4ace-b809-33dedeff226a admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:49.185016 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ed5f07-8d95-4ace-b809-33dedeff226a admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:49.196871 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ed5f07-8d95-4ace-b809-33dedeff226a admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 00:01:49.201878 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-95ed5f07-8d95-4ace-b809-33dedeff226a admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 12 00:01:49.201878 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:01:49.201878 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:01:49.201878 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:01:49.201878 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:01:49.201878 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:01:49.201878 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:01:49.201878 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:01:49.201878 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:01:49.201878 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:01:49.201878 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:01:49.201878 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:01:49.201878 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 00:01:49.201878 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 00:01:49.201878 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 00:01:49.201878 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 00:01:49.202722 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 00:01:49.202722 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 00:01:49.202722 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 00:01:49.202722 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 00:01:49.202722 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:01:49.202722 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:01:49.202722 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:01:49.202722 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:01:49.202722 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:01:49.202722 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:01:49.202722 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:01:49.202722 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:01:49.202722 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:01:49.202722 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:01:49.202722 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:01:49.202722 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:01:49.204245 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:01:49.204245 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:01:49.204245 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:01:49.204245 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:01:49.204245 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 00:01:49.204245 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 00:01:49.204245 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 00:01:49.204245 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 00:01:49.204245 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 00:01:49.204245 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 00:01:49.204245 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 12 00:01:49.204245 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:01:49.204245 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 135/274] 10.222.0.36 () {56 vars in 1120 bytes} [Wed Oct 12 00:01:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 112 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 00:01:49.209621 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177392dc-e7bd-481c-b779-dabb82f22a2d None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:49.226210 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177392dc-e7bd-481c-b779-dabb82f22a2d None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:49.233844 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc20b37d-e748-4d57-927e-e2cdd7ab8cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:49.234574 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc20b37d-e748-4d57-927e-e2cdd7ab8cb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:49.234853 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc20b37d-e748-4d57-927e-e2cdd7ab8cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:49.235123 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc20b37d-e748-4d57-927e-e2cdd7ab8cb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:49.236466 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc20b37d-e748-4d57-927e-e2cdd7ab8cb5 admin admin] RBAC: Adding query filter params (name=member) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:01:49.236466 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc20b37d-e748-4d57-927e-e2cdd7ab8cb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:49.238858 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc20b37d-e748-4d57-927e-e2cdd7ab8cb5 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:49.252611 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 140/275] 10.222.0.36 () {56 vars in 1122 bytes} [Wed Oct 12 00:01:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:49.258450 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ed5f07-8d95-4ace-b809-33dedeff226a admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:49.269166 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ed5f07-8d95-4ace-b809-33dedeff226a admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:49.279581 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8428e09d-4200-4edf-a06c-cfbd4bc8c16d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:49.280230 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428e09d-4200-4edf-a06c-cfbd4bc8c16d admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:49.280506 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428e09d-4200-4edf-a06c-cfbd4bc8c16d admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:49.280777 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428e09d-4200-4edf-a06c-cfbd4bc8c16d admin admin] PATH_INFO: `/v3/groups/519fd3c5007f452e8c09b26b1c49ccce` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:49.288049 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8428e09d-4200-4edf-a06c-cfbd4bc8c16d admin admin] RBAC: Authorizing `identity:get_group(group_id=519fd3c5007f452e8c09b26b1c49ccce)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:49.289092 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: WARNING py.warnings [None req-8428e09d-4200-4edf-a06c-cfbd4bc8c16d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 12 00:01:49.289092 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: warnings.warn(msg) Oct 12 00:01:49.289092 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: Oct 12 00:01:49.290309 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8428e09d-4200-4edf-a06c-cfbd4bc8c16d admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:49.292259 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 136/276] 10.222.0.36 () {56 vars in 1201 bytes} [Wed Oct 12 00:01:49 2022] GET /identity/v3/groups/519fd3c5007f452e8c09b26b1c49ccce => generated 226 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:49.297836 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc20b37d-e748-4d57-927e-e2cdd7ab8cb5 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:49.312577 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc20b37d-e748-4d57-927e-e2cdd7ab8cb5 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:49.319257 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a910307f-5379-489b-bb56-e2c7d180ea74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:49.320090 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a910307f-5379-489b-bb56-e2c7d180ea74 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:49.320366 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a910307f-5379-489b-bb56-e2c7d180ea74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:49.320695 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a910307f-5379-489b-bb56-e2c7d180ea74 admin admin] PATH_INFO: `/v3/projects/25ba77a2f84b41cd99a774b93ada5e96` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:49.323261 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a910307f-5379-489b-bb56-e2c7d180ea74 admin admin] RBAC: Authorizing `identity:get_project(project_id=25ba77a2f84b41cd99a774b93ada5e96)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:49.325496 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a910307f-5379-489b-bb56-e2c7d180ea74 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:49.327498 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 141/277] 10.222.0.36 () {56 vars in 1207 bytes} [Wed Oct 12 00:01:49 2022] GET /identity/v3/projects/25ba77a2f84b41cd99a774b93ada5e96 => generated 302 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:49.333903 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8428e09d-4200-4edf-a06c-cfbd4bc8c16d admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:49.349159 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8428e09d-4200-4edf-a06c-cfbd4bc8c16d admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:49.359505 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1ea699-9346-4ac5-af08-7ccdcd6effc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:49.360183 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1ea699-9346-4ac5-af08-7ccdcd6effc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:49.360459 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1ea699-9346-4ac5-af08-7ccdcd6effc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:49.360730 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1ea699-9346-4ac5-af08-7ccdcd6effc0 admin admin] PATH_INFO: `/v3/projects/25ba77a2f84b41cd99a774b93ada5e96/groups/519fd3c5007f452e8c09b26b1c49ccce/roles/ef689f96442945e8a7bc8cbfee13d909` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:49.364753 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1ea699-9346-4ac5-af08-7ccdcd6effc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25ba77a2f84b41cd99a774b93ada5e96, group_id=519fd3c5007f452e8c09b26b1c49ccce, role_id=ef689f96442945e8a7bc8cbfee13d909)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:49.367461 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1ea699-9346-4ac5-af08-7ccdcd6effc0 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:49.463238 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 137/278] 10.222.0.36 () {58 vars in 1463 bytes} [Wed Oct 12 00:01:49 2022] PUT /identity/v3/projects/25ba77a2f84b41cd99a774b93ada5e96/groups/519fd3c5007f452e8c09b26b1c49ccce/roles/ef689f96442945e8a7bc8cbfee13d909 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:01:49.548089 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a910307f-5379-489b-bb56-e2c7d180ea74 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:49.667629 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a910307f-5379-489b-bb56-e2c7d180ea74 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:49.708455 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19270759-fa5e-4054-b398-be4c3b04638c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:49.709097 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19270759-fa5e-4054-b398-be4c3b04638c None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:49.709373 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19270759-fa5e-4054-b398-be4c3b04638c None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:49.709642 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19270759-fa5e-4054-b398-be4c3b04638c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:49.721481 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19270759-fa5e-4054-b398-be4c3b04638c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 00:01:49.727762 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-19270759-fa5e-4054-b398-be4c3b04638c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 12 00:01:49.727762 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:01:49.727762 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:01:49.727762 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:01:49.727762 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:01:49.727762 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:01:49.727762 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:01:49.727762 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:01:49.727762 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:01:49.727762 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:01:49.727762 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:01:49.727762 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:01:49.727762 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 00:01:49.727762 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 00:01:49.727762 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 00:01:49.727762 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 00:01:49.728645 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 00:01:49.728645 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 00:01:49.728645 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 00:01:49.728645 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 00:01:49.728645 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:01:49.728645 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:01:49.728645 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:01:49.728645 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:01:49.728645 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:01:49.728645 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:01:49.728645 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:01:49.728645 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:01:49.728645 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:01:49.728645 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:01:49.728645 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:01:49.728645 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:01:49.729996 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:01:49.729996 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:01:49.729996 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:01:49.729996 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:01:49.729996 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 00:01:49.729996 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 00:01:49.729996 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 00:01:49.729996 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 00:01:49.729996 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 00:01:49.729996 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 00:01:49.729996 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 12 00:01:49.729996 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:01:49.729996 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 142/279] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Oct 12 00:01:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:01:49.734771 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1ea699-9346-4ac5-af08-7ccdcd6effc0 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:49.861242 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1ea699-9346-4ac5-af08-7ccdcd6effc0 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:49.909092 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fc30ae-21f8-4401-8525-4f3c6c60743e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:49.909943 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fc30ae-21f8-4401-8525-4f3c6c60743e None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:49.910416 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fc30ae-21f8-4401-8525-4f3c6c60743e None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:49.910893 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fc30ae-21f8-4401-8525-4f3c6c60743e None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:49.911728 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fc30ae-21f8-4401-8525-4f3c6c60743e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:01:49.912170 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fc30ae-21f8-4401-8525-4f3c6c60743e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:49.913695 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fc30ae-21f8-4401-8525-4f3c6c60743e None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:49.923980 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 138/280] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Oct 12 00:01:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 00:01:49.929339 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19270759-fa5e-4054-b398-be4c3b04638c None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:49.951165 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:50.042749 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19270759-fa5e-4054-b398-be4c3b04638c None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:50.073590 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7b90bb-6a8d-46ac-bff3-0ac1e266d043 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:50.074096 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7b90bb-6a8d-46ac-bff3-0ac1e266d043 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:50.074096 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7b90bb-6a8d-46ac-bff3-0ac1e266d043 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:50.075731 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 139/281] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:01:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 00:01:50.082009 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59577614-157f-46bf-a0b1-128508d4afaa None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:50.082409 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59577614-157f-46bf-a0b1-128508d4afaa None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:50.083400 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59577614-157f-46bf-a0b1-128508d4afaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:50.086906 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d524aa-78b9-4439-9707-b779a3c6726e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:50.087755 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d524aa-78b9-4439-9707-b779a3c6726e None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:50.088157 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d524aa-78b9-4439-9707-b779a3c6726e None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:50.088552 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d524aa-78b9-4439-9707-b779a3c6726e None admin] PATH_INFO: `/v3/users/8d1992b609284eb5b9d51c1a17b4c4d4` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:50.097278 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d524aa-78b9-4439-9707-b779a3c6726e None admin] RBAC: Authorizing `identity:get_user(user_id=8d1992b609284eb5b9d51c1a17b4c4d4)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:50.100179 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d524aa-78b9-4439-9707-b779a3c6726e None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:50.101911 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 143/282] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Oct 12 00:01:49 2022] GET /identity/v3/users/8d1992b609284eb5b9d51c1a17b4c4d4 => generated 293 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 00:01:50.110275 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d524aa-78b9-4439-9707-b779a3c6726e None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:50.114910 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-59577614-157f-46bf-a0b1-128508d4afaa None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:01:50.116090 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-59577614-157f-46bf-a0b1-128508d4afaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:01:50.141938 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:50.204662 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 140/283] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:01:50 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 12 00:01:50.236271 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:50.260126 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d524aa-78b9-4439-9707-b779a3c6726e None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:50.300986 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7ac721-ac7e-4fcf-9e78-e4ed71137ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:50.302134 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7ac721-ac7e-4fcf-9e78-e4ed71137ea1 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:50.302411 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7ac721-ac7e-4fcf-9e78-e4ed71137ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:50.302682 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7ac721-ac7e-4fcf-9e78-e4ed71137ea1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:50.303425 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7ac721-ac7e-4fcf-9e78-e4ed71137ea1 None admin] RBAC: Adding query filter params (role.id=217d497bae80496a9d3772fbb90678e1, user.id=8d1992b609284eb5b9d51c1a17b4c4d4, scope.system=all) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:01:50.303736 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7ac721-ac7e-4fcf-9e78-e4ed71137ea1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:50.305173 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7ac721-ac7e-4fcf-9e78-e4ed71137ea1 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:50.318198 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 144/284] 10.222.0.36 () {56 vars in 1308 bytes} [Wed Oct 12 00:01:50 2022] GET /identity/v3/role_assignments?user.id=8d1992b609284eb5b9d51c1a17b4c4d4&scope.system=all&role.id=217d497bae80496a9d3772fbb90678e1 => generated 510 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:50.570075 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59577614-157f-46bf-a0b1-128508d4afaa None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:50.581268 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59577614-157f-46bf-a0b1-128508d4afaa None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:50.587585 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f00897-ec52-4bcb-a63f-b37a44052dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:50.588240 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f00897-ec52-4bcb-a63f-b37a44052dc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:50.588511 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f00897-ec52-4bcb-a63f-b37a44052dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:50.588775 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f00897-ec52-4bcb-a63f-b37a44052dc3 admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:50.603017 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f00897-ec52-4bcb-a63f-b37a44052dc3 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 00:01:50.608108 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-26f00897-ec52-4bcb-a63f-b37a44052dc3 admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 12 00:01:50.608108 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:01:50.608108 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:01:50.608108 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:01:50.608108 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:01:50.608108 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:01:50.608108 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:01:50.608108 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:01:50.608108 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:01:50.608108 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:01:50.608108 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:01:50.608108 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:01:50.608108 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 00:01:50.608108 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 00:01:50.608108 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 00:01:50.608108 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 00:01:50.609041 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 00:01:50.609041 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 00:01:50.609041 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 00:01:50.609041 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 00:01:50.609041 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:01:50.609041 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:01:50.609041 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:01:50.609041 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:01:50.609041 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:01:50.609041 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:01:50.609041 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:01:50.609041 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:01:50.609041 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:01:50.609041 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:01:50.609041 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:01:50.609041 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:01:50.609813 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:01:50.609813 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:01:50.609813 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:01:50.609813 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:01:50.609813 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 00:01:50.609813 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 00:01:50.609813 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 00:01:50.609813 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 00:01:50.609813 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 00:01:50.609813 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 00:01:50.609813 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 12 00:01:50.609813 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:01:50.609813 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 141/285] 10.222.0.36 () {56 vars in 1135 bytes} [Wed Oct 12 00:01:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 40 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:01:50.615905 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7ac721-ac7e-4fcf-9e78-e4ed71137ea1 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:50.631861 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7ac721-ac7e-4fcf-9e78-e4ed71137ea1 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:50.632785 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c88df6-3b81-459c-b9f4-b32d7f482c4b None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:50.633070 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c88df6-3b81-459c-b9f4-b32d7f482c4b None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:50.633342 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c88df6-3b81-459c-b9f4-b32d7f482c4b None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:50.634937 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 142/286] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:01:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:01:50.640615 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c468ff25-f2ef-4844-bf54-14f4e831af20 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:50.640861 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c468ff25-f2ef-4844-bf54-14f4e831af20 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:50.641064 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30383448-c5e6-41cd-91c4-3889f71e182d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:50.641263 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c468ff25-f2ef-4844-bf54-14f4e831af20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:50.641706 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30383448-c5e6-41cd-91c4-3889f71e182d admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:50.641956 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30383448-c5e6-41cd-91c4-3889f71e182d admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:50.642227 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30383448-c5e6-41cd-91c4-3889f71e182d admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:50.642909 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30383448-c5e6-41cd-91c4-3889f71e182d admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:01:50.643179 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30383448-c5e6-41cd-91c4-3889f71e182d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:50.644796 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30383448-c5e6-41cd-91c4-3889f71e182d admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:50.664553 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 145/287] 10.222.0.36 () {56 vars in 1132 bytes} [Wed Oct 12 00:01:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:50.670116 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30383448-c5e6-41cd-91c4-3889f71e182d admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:50.678132 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-c468ff25-f2ef-4844-bf54-14f4e831af20 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:01:50.679690 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-c468ff25-f2ef-4844-bf54-14f4e831af20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:01:50.684930 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30383448-c5e6-41cd-91c4-3889f71e182d admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:50.694393 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dca98f-6919-467e-b22c-263ae7aae836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:50.696526 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dca98f-6919-467e-b22c-263ae7aae836 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:50.696526 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dca98f-6919-467e-b22c-263ae7aae836 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:50.696526 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dca98f-6919-467e-b22c-263ae7aae836 admin admin] PATH_INFO: `/v3/groups/519fd3c5007f452e8c09b26b1c49ccce` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:50.696995 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 143/288] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:01:50 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 12 00:01:50.697138 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dca98f-6919-467e-b22c-263ae7aae836 admin admin] RBAC: Authorizing `identity:get_group(group_id=519fd3c5007f452e8c09b26b1c49ccce)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:50.699334 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dca98f-6919-467e-b22c-263ae7aae836 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:50.704810 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 146/289] 10.222.0.36 () {56 vars in 1201 bytes} [Wed Oct 12 00:01:50 2022] GET /identity/v3/groups/519fd3c5007f452e8c09b26b1c49ccce => generated 226 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 00:01:50.705945 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c468ff25-f2ef-4844-bf54-14f4e831af20 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:50.715895 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c468ff25-f2ef-4844-bf54-14f4e831af20 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:50.724209 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b64377-2c4e-49bf-967d-06d9d63e30e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:50.725188 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b64377-2c4e-49bf-967d-06d9d63e30e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:50.725701 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b64377-2c4e-49bf-967d-06d9d63e30e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:50.726241 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b64377-2c4e-49bf-967d-06d9d63e30e2 admin admin] PATH_INFO: `/v3/projects/25ba77a2f84b41cd99a774b93ada5e96` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:50.727624 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b64377-2c4e-49bf-967d-06d9d63e30e2 admin admin] RBAC: Authorizing `identity:get_project(project_id=25ba77a2f84b41cd99a774b93ada5e96)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:50.730163 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b64377-2c4e-49bf-967d-06d9d63e30e2 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:50.732283 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 144/290] 10.222.0.36 () {56 vars in 1207 bytes} [Wed Oct 12 00:01:50 2022] GET /identity/v3/projects/25ba77a2f84b41cd99a774b93ada5e96 => generated 302 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:50.740767 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dca98f-6919-467e-b22c-263ae7aae836 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:50.754870 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dca98f-6919-467e-b22c-263ae7aae836 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:50.763858 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4b54ca-5a89-4ae1-a7c5-2e26c5c8ee93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:50.764521 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b54ca-5a89-4ae1-a7c5-2e26c5c8ee93 admin admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:50.764796 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b54ca-5a89-4ae1-a7c5-2e26c5c8ee93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:50.765063 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b54ca-5a89-4ae1-a7c5-2e26c5c8ee93 admin admin] PATH_INFO: `/v3/projects/25ba77a2f84b41cd99a774b93ada5e96/groups/519fd3c5007f452e8c09b26b1c49ccce/roles/81991615ac0b46818426bf37015df930` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:50.771770 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4b54ca-5a89-4ae1-a7c5-2e26c5c8ee93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25ba77a2f84b41cd99a774b93ada5e96, group_id=519fd3c5007f452e8c09b26b1c49ccce, role_id=81991615ac0b46818426bf37015df930)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:50.774400 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4b54ca-5a89-4ae1-a7c5-2e26c5c8ee93 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:50.862896 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 147/291] 10.222.0.36 () {58 vars in 1463 bytes} [Wed Oct 12 00:01:50 2022] PUT /identity/v3/projects/25ba77a2f84b41cd99a774b93ada5e96/groups/519fd3c5007f452e8c09b26b1c49ccce/roles/81991615ac0b46818426bf37015df930 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:01:51.095961 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b64377-2c4e-49bf-967d-06d9d63e30e2 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:51.108102 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b64377-2c4e-49bf-967d-06d9d63e30e2 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:51.179144 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8279fe88-5d0b-4bd4-aa93-23aa76d2ba1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:51.179749 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8279fe88-5d0b-4bd4-aa93-23aa76d2ba1d admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:51.180026 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8279fe88-5d0b-4bd4-aa93-23aa76d2ba1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:51.180297 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8279fe88-5d0b-4bd4-aa93-23aa76d2ba1d admin admin] PATH_INFO: `/v3/roles/admin` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:51.191723 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8279fe88-5d0b-4bd4-aa93-23aa76d2ba1d admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 00:01:51.197171 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-8279fe88-5d0b-4bd4-aa93-23aa76d2ba1d admin admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 00:01:51.197171 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:01:51.197171 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:01:51.197171 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:01:51.197171 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:01:51.197171 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:01:51.197171 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:01:51.197171 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:01:51.197171 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:01:51.197171 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:01:51.197171 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:01:51.197171 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:01:51.197171 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 00:01:51.197171 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 00:01:51.197171 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 00:01:51.197171 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 00:01:51.198052 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 00:01:51.198052 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 00:01:51.198052 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 00:01:51.198052 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 00:01:51.198052 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:01:51.198052 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:01:51.198052 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:01:51.198052 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:01:51.198052 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:01:51.198052 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:01:51.198052 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:01:51.198052 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:01:51.198052 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:01:51.198052 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:01:51.198052 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:01:51.198052 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:01:51.198836 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:01:51.198836 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:01:51.198836 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:01:51.198836 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:01:51.198836 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 00:01:51.198836 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 00:01:51.198836 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 00:01:51.198836 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 00:01:51.198836 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 00:01:51.198836 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 00:01:51.198836 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 00:01:51.198836 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:01:51.198836 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 145/292] 10.222.0.36 () {56 vars in 1117 bytes} [Wed Oct 12 00:01:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 103 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:01:51.204346 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4b54ca-5a89-4ae1-a7c5-2e26c5c8ee93 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:51.217598 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4b54ca-5a89-4ae1-a7c5-2e26c5c8ee93 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:51.222457 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e857de96-e58b-453c-8ce7-0fc4450a1cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:51.223067 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e857de96-e58b-453c-8ce7-0fc4450a1cae admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:51.223929 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e857de96-e58b-453c-8ce7-0fc4450a1cae admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:51.224201 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e857de96-e58b-453c-8ce7-0fc4450a1cae admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:51.224867 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e857de96-e58b-453c-8ce7-0fc4450a1cae admin admin] RBAC: Adding query filter params (name=admin) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:01:51.225167 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e857de96-e58b-453c-8ce7-0fc4450a1cae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:51.226823 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e857de96-e58b-453c-8ce7-0fc4450a1cae admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:51.236629 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 148/293] 10.222.0.36 () {56 vars in 1120 bytes} [Wed Oct 12 00:01:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:51.242917 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8279fe88-5d0b-4bd4-aa93-23aa76d2ba1d admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:51.256917 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8279fe88-5d0b-4bd4-aa93-23aa76d2ba1d admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:51.262182 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d2377f-8849-4799-a06c-e95ef489824b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:51.262864 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2377f-8849-4799-a06c-e95ef489824b admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:51.263147 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2377f-8849-4799-a06c-e95ef489824b admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:51.263439 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2377f-8849-4799-a06c-e95ef489824b admin admin] PATH_INFO: `/v3/groups/f51f9cce250a4b209eabc1ea9630524f` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:51.271305 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d2377f-8849-4799-a06c-e95ef489824b admin admin] RBAC: Authorizing `identity:get_group(group_id=f51f9cce250a4b209eabc1ea9630524f)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:51.273376 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d2377f-8849-4799-a06c-e95ef489824b admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:51.275547 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 146/294] 10.222.0.36 () {56 vars in 1201 bytes} [Wed Oct 12 00:01:51 2022] GET /identity/v3/groups/f51f9cce250a4b209eabc1ea9630524f => generated 229 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:51.281271 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e857de96-e58b-453c-8ce7-0fc4450a1cae admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:51.292628 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e857de96-e58b-453c-8ce7-0fc4450a1cae admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:51.297187 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fc98dc-e0e5-4805-b299-f6782988a611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:51.297821 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc98dc-e0e5-4805-b299-f6782988a611 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:51.298156 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc98dc-e0e5-4805-b299-f6782988a611 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:51.298430 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc98dc-e0e5-4805-b299-f6782988a611 admin admin] PATH_INFO: `/v3/projects/392dba9b727b479aa54a2746402ded73` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:51.299686 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fc98dc-e0e5-4805-b299-f6782988a611 admin admin] RBAC: Authorizing `identity:get_project(project_id=392dba9b727b479aa54a2746402ded73)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:51.302150 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fc98dc-e0e5-4805-b299-f6782988a611 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:51.304202 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 149/295] 10.222.0.36 () {56 vars in 1207 bytes} [Wed Oct 12 00:01:51 2022] GET /identity/v3/projects/392dba9b727b479aa54a2746402ded73 => generated 344 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:51.309897 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d2377f-8849-4799-a06c-e95ef489824b admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:51.320902 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d2377f-8849-4799-a06c-e95ef489824b admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:51.325550 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df971327-aae5-441a-9d34-2f5e50e511cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:51.326264 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df971327-aae5-441a-9d34-2f5e50e511cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:51.326545 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df971327-aae5-441a-9d34-2f5e50e511cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:51.326819 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df971327-aae5-441a-9d34-2f5e50e511cc admin admin] PATH_INFO: `/v3/projects/392dba9b727b479aa54a2746402ded73/groups/f51f9cce250a4b209eabc1ea9630524f/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:51.328921 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df971327-aae5-441a-9d34-2f5e50e511cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=392dba9b727b479aa54a2746402ded73, group_id=f51f9cce250a4b209eabc1ea9630524f, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:51.331761 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df971327-aae5-441a-9d34-2f5e50e511cc admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:51.410441 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 147/296] 10.222.0.36 () {58 vars in 1463 bytes} [Wed Oct 12 00:01:51 2022] PUT /identity/v3/projects/392dba9b727b479aa54a2746402ded73/groups/f51f9cce250a4b209eabc1ea9630524f/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:01:51.991030 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdbe623-6900-4361-a338-9f294aa3bc7a None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:51.991828 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdbe623-6900-4361-a338-9f294aa3bc7a None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:51.992332 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdbe623-6900-4361-a338-9f294aa3bc7a None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:51.993900 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 150/297] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:01:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:01:51.999941 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec08c0e0-90f7-478e-85f3-9715ed1d7d6a None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:52.000585 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec08c0e0-90f7-478e-85f3-9715ed1d7d6a None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:52.001083 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec08c0e0-90f7-478e-85f3-9715ed1d7d6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:52.027844 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-ec08c0e0-90f7-478e-85f3-9715ed1d7d6a None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:01:52.029093 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-ec08c0e0-90f7-478e-85f3-9715ed1d7d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:01:52.097739 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:52.108971 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 148/298] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:01:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 00:01:52.227970 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff7489-8325-41c9-a9cc-9f75b5f67e9c None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:52.228479 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff7489-8325-41c9-a9cc-9f75b5f67e9c None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:52.228549 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff7489-8325-41c9-a9cc-9f75b5f67e9c None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:52.229838 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 151/299] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:01:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:01:52.245334 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e8a98a-fe6b-44a3-843f-8b72cf0b8c4c None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:52.245593 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e8a98a-fe6b-44a3-843f-8b72cf0b8c4c None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:52.245817 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e8a98a-fe6b-44a3-843f-8b72cf0b8c4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:52.269862 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-a9e8a98a-fe6b-44a3-843f-8b72cf0b8c4c None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:01:52.271636 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-a9e8a98a-fe6b-44a3-843f-8b72cf0b8c4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:01:52.343777 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 149/300] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:01:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 00:01:52.578951 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dff7489-8325-41c9-a9cc-9f75b5f67e9c None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:52.709588 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dff7489-8325-41c9-a9cc-9f75b5f67e9c None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:52.711280 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:52.756311 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f85a0ad-1520-4aea-9d22-c5f9cfef4668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:52.756998 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f85a0ad-1520-4aea-9d22-c5f9cfef4668 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:52.757277 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f85a0ad-1520-4aea-9d22-c5f9cfef4668 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:52.757548 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f85a0ad-1520-4aea-9d22-c5f9cfef4668 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:52.772593 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f85a0ad-1520-4aea-9d22-c5f9cfef4668 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 00:01:52.777659 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-2f85a0ad-1520-4aea-9d22-c5f9cfef4668 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 12 00:01:52.777659 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:01:52.777659 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:01:52.777659 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:01:52.777659 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:01:52.777659 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:01:52.777659 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:01:52.777659 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:01:52.777659 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:01:52.777659 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:01:52.777659 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:01:52.777659 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:01:52.777659 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 00:01:52.777659 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 00:01:52.777659 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 00:01:52.777659 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 00:01:52.782843 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 00:01:52.782843 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 00:01:52.782843 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 00:01:52.782843 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 00:01:52.782843 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:01:52.782843 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:01:52.782843 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:01:52.782843 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:01:52.782843 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:01:52.782843 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:01:52.782843 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:01:52.782843 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:01:52.782843 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:01:52.782843 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:01:52.782843 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:01:52.782843 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:01:52.784620 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:01:52.784620 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:01:52.784620 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:01:52.784620 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:01:52.784620 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 00:01:52.784620 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 00:01:52.784620 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 00:01:52.784620 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 00:01:52.784620 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 00:01:52.784620 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 00:01:52.784620 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 12 00:01:52.784620 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:01:52.784620 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 152/301] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Oct 12 00:01:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:01:52.785282 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e8a98a-fe6b-44a3-843f-8b72cf0b8c4c None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:52.884358 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a68d273-2a2f-418d-9b55-dfd670da80ca None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:52.884937 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a68d273-2a2f-418d-9b55-dfd670da80ca None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:52.885218 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a68d273-2a2f-418d-9b55-dfd670da80ca None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:52.886815 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 153/302] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:01:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:01:52.891432 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9b1528-e64f-43ef-afdc-cde7592940e1 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:52.891709 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9b1528-e64f-43ef-afdc-cde7592940e1 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:52.891985 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9b1528-e64f-43ef-afdc-cde7592940e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:52.913561 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-aa9b1528-e64f-43ef-afdc-cde7592940e1 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:01:52.914791 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-aa9b1528-e64f-43ef-afdc-cde7592940e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:01:52.937024 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e8a98a-fe6b-44a3-843f-8b72cf0b8c4c None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:52.970787 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:52.994254 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6fe373-8a27-45bc-ad1d-d8ced145cbd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:52.995283 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6fe373-8a27-45bc-ad1d-d8ced145cbd3 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:52.995939 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6fe373-8a27-45bc-ad1d-d8ced145cbd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:52.996347 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6fe373-8a27-45bc-ad1d-d8ced145cbd3 None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:52.997087 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6fe373-8a27-45bc-ad1d-d8ced145cbd3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:01:52.997406 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6fe373-8a27-45bc-ad1d-d8ced145cbd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:52.998786 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6fe373-8a27-45bc-ad1d-d8ced145cbd3 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:53.000791 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 154/303] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:01:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 12 00:01:53.001465 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9b1528-e64f-43ef-afdc-cde7592940e1 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:53.009222 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 150/304] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Oct 12 00:01:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:53.014503 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6fe373-8a27-45bc-ad1d-d8ced145cbd3 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:53.159103 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6fe373-8a27-45bc-ad1d-d8ced145cbd3 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:53.179036 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:53.189072 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9b1528-e64f-43ef-afdc-cde7592940e1 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:53.232717 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ade734d-f44b-466c-9688-19bd3f0f199f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:53.233660 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ade734d-f44b-466c-9688-19bd3f0f199f None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:53.234092 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ade734d-f44b-466c-9688-19bd3f0f199f None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:53.237550 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ade734d-f44b-466c-9688-19bd3f0f199f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:53.244202 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d301375-fd31-46af-9e00-87f02061c022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:53.248857 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d301375-fd31-46af-9e00-87f02061c022 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:53.249423 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d301375-fd31-46af-9e00-87f02061c022 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:53.249891 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d301375-fd31-46af-9e00-87f02061c022 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:53.276863 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d301375-fd31-46af-9e00-87f02061c022 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 00:01:53.289044 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-9d301375-fd31-46af-9e00-87f02061c022 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 12 00:01:53.289044 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:01:53.289044 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:01:53.289044 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:01:53.289044 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:01:53.289044 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:01:53.289044 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:01:53.289044 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:01:53.289044 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:01:53.289044 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:01:53.289044 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:01:53.289044 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:01:53.289044 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 00:01:53.289044 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 00:01:53.289044 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 00:01:53.289044 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 00:01:53.289937 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 00:01:53.289937 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 00:01:53.289937 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 00:01:53.289937 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 00:01:53.289937 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:01:53.289937 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:01:53.289937 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:01:53.289937 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:01:53.289937 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:01:53.289937 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:01:53.289937 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:01:53.289937 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:01:53.289937 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:01:53.289937 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:01:53.289937 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:01:53.289937 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:01:53.290719 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:01:53.290719 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:01:53.290719 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:01:53.290719 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:01:53.290719 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 00:01:53.290719 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 00:01:53.290719 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 00:01:53.290719 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 00:01:53.290719 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 00:01:53.290719 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 00:01:53.290719 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 12 00:01:53.290719 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:01:53.295268 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 155/305] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Oct 12 00:01:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:01:53.300397 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d301375-fd31-46af-9e00-87f02061c022 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:53.342660 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:53.368395 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ade734d-f44b-466c-9688-19bd3f0f199f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:53.370395 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ade734d-f44b-466c-9688-19bd3f0f199f None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:53.441570 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d301375-fd31-46af-9e00-87f02061c022 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:53.484067 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f50e36-a751-410d-b1cc-9a008657a896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:53.484681 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f50e36-a751-410d-b1cc-9a008657a896 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:53.484955 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f50e36-a751-410d-b1cc-9a008657a896 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:53.485224 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f50e36-a751-410d-b1cc-9a008657a896 None admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:53.486049 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f50e36-a751-410d-b1cc-9a008657a896 None admin] RBAC: Adding query filter params (name=member) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:01:53.486355 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f50e36-a751-410d-b1cc-9a008657a896 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:53.488068 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f50e36-a751-410d-b1cc-9a008657a896 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:53.499716 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 156/306] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Oct 12 00:01:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:53.499716 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f50e36-a751-410d-b1cc-9a008657a896 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:53.513211 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 151/307] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Oct 12 00:01:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 12 00:01:53.514414 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ade734d-f44b-466c-9688-19bd3f0f199f None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:53.523657 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:53.649923 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:53.653279 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f50e36-a751-410d-b1cc-9a008657a896 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:53.678030 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ade734d-f44b-466c-9688-19bd3f0f199f None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:53.703624 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0534729-0cd2-4105-8589-13079bdad1da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:53.704651 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0534729-0cd2-4105-8589-13079bdad1da None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:53.705133 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0534729-0cd2-4105-8589-13079bdad1da None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:53.705624 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0534729-0cd2-4105-8589-13079bdad1da None admin] PATH_INFO: `/v3/roles/member` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:53.722833 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0534729-0cd2-4105-8589-13079bdad1da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 00:01:53.725061 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a55a35-b901-4624-905c-7e9dc6864af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:53.725738 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a55a35-b901-4624-905c-7e9dc6864af3 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:53.726082 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a55a35-b901-4624-905c-7e9dc6864af3 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:53.726417 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a55a35-b901-4624-905c-7e9dc6864af3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:53.731320 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-e0534729-0cd2-4105-8589-13079bdad1da None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 12 00:01:53.731320 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:01:53.731320 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:01:53.731320 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:01:53.731320 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:01:53.731320 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:01:53.731320 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:01:53.731320 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:01:53.731320 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:01:53.731320 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:01:53.731320 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:01:53.731320 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:01:53.731320 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 00:01:53.731320 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 00:01:53.731320 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 00:01:53.731320 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 00:01:53.732268 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 00:01:53.732268 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 00:01:53.732268 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 00:01:53.732268 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 00:01:53.732268 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:01:53.732268 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:01:53.732268 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:01:53.732268 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:01:53.732268 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:01:53.732268 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:01:53.732268 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:01:53.732268 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:01:53.732268 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:01:53.732268 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:01:53.732268 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:01:53.732268 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:01:53.733043 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:01:53.733043 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:01:53.733043 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:01:53.733043 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:01:53.733043 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 00:01:53.733043 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 00:01:53.733043 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 00:01:53.733043 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 00:01:53.733043 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 00:01:53.733043 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 00:01:53.733043 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 12 00:01:53.733043 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:01:53.735759 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 157/308] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Oct 12 00:01:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:01:53.737317 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0534729-0cd2-4105-8589-13079bdad1da None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:53.819614 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a55a35-b901-4624-905c-7e9dc6864af3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:53.821123 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a55a35-b901-4624-905c-7e9dc6864af3 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:53.841742 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:53.873510 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0534729-0cd2-4105-8589-13079bdad1da None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:53.915983 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72e6cef-a180-46ec-826c-6d598b06ed17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:53.916633 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72e6cef-a180-46ec-826c-6d598b06ed17 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:53.916912 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72e6cef-a180-46ec-826c-6d598b06ed17 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:53.917185 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72e6cef-a180-46ec-826c-6d598b06ed17 None admin] PATH_INFO: `/v3/projects/410027d6c764410ea5b76db0171903df` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:53.918588 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72e6cef-a180-46ec-826c-6d598b06ed17 None admin] RBAC: Authorizing `identity:get_project(project_id=410027d6c764410ea5b76db0171903df)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:53.920097 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72e6cef-a180-46ec-826c-6d598b06ed17 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:53.921910 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 158/309] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Oct 12 00:01:53 2022] GET /identity/v3/projects/410027d6c764410ea5b76db0171903df => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:53.923237 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72e6cef-a180-46ec-826c-6d598b06ed17 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:53.927478 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:53.955099 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 152/310] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Oct 12 00:01:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 12 00:01:53.956147 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd3c0f9-1ba0-4f4a-a84f-5210cafa6640 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:53.956460 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd3c0f9-1ba0-4f4a-a84f-5210cafa6640 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:53.956780 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd3c0f9-1ba0-4f4a-a84f-5210cafa6640 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:53.962740 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 153/311] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:01:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:01:53.962740 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd3c0f9-1ba0-4f4a-a84f-5210cafa6640 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:54.052396 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72e6cef-a180-46ec-826c-6d598b06ed17 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:54.083424 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd3c0f9-1ba0-4f4a-a84f-5210cafa6640 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:54.093057 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b01619a-119d-4683-9a0f-b0c1b8c4e0e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:54.093788 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b01619a-119d-4683-9a0f-b0c1b8c4e0e9 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:54.094067 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b01619a-119d-4683-9a0f-b0c1b8c4e0e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:54.094333 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b01619a-119d-4683-9a0f-b0c1b8c4e0e9 None admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:54.096139 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b01619a-119d-4683-9a0f-b0c1b8c4e0e9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:01:54.096543 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b01619a-119d-4683-9a0f-b0c1b8c4e0e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:54.098691 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b01619a-119d-4683-9a0f-b0c1b8c4e0e9 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:54.109172 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 159/312] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Oct 12 00:01:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:54.110167 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b01619a-119d-4683-9a0f-b0c1b8c4e0e9 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:54.113737 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:54.133050 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3709988a-e444-4031-ae79-ca9c841ac6a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:54.133695 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3709988a-e444-4031-ae79-ca9c841ac6a4 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:54.133969 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3709988a-e444-4031-ae79-ca9c841ac6a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:54.134241 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3709988a-e444-4031-ae79-ca9c841ac6a4 None admin] PATH_INFO: `/v3/groups/519fd3c5007f452e8c09b26b1c49ccce` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:54.135687 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3709988a-e444-4031-ae79-ca9c841ac6a4 None admin] RBAC: Authorizing `identity:get_group(group_id=519fd3c5007f452e8c09b26b1c49ccce)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:54.137142 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3709988a-e444-4031-ae79-ca9c841ac6a4 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:54.139013 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 154/313] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Oct 12 00:01:53 2022] GET /identity/v3/groups/519fd3c5007f452e8c09b26b1c49ccce => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:54.140523 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d9ee2c-e4b6-49ec-aeee-a8fa87761d77 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:54.140831 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d9ee2c-e4b6-49ec-aeee-a8fa87761d77 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:54.141082 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d9ee2c-e4b6-49ec-aeee-a8fa87761d77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:54.162785 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-c3d9ee2c-e4b6-49ec-aeee-a8fa87761d77 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:01:54.164305 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-c3d9ee2c-e4b6-49ec-aeee-a8fa87761d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:01:54.219112 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:54.239925 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 155/314] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:01:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 00:01:54.244156 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b01619a-119d-4683-9a0f-b0c1b8c4e0e9 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:54.244291 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d9ee2c-e4b6-49ec-aeee-a8fa87761d77 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:54.286118 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3ac57f-1c6f-430c-908c-d8765b9f2f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:54.286787 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3ac57f-1c6f-430c-908c-d8765b9f2f80 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:54.287046 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3ac57f-1c6f-430c-908c-d8765b9f2f80 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:54.287319 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3ac57f-1c6f-430c-908c-d8765b9f2f80 None admin] PATH_INFO: `/v3/projects/410027d6c764410ea5b76db0171903df` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:54.290509 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3ac57f-1c6f-430c-908c-d8765b9f2f80 None admin] RBAC: Authorizing `identity:get_project(project_id=410027d6c764410ea5b76db0171903df)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:54.291390 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3ac57f-1c6f-430c-908c-d8765b9f2f80 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:54.293277 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 160/315] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Oct 12 00:01:54 2022] GET /identity/v3/projects/410027d6c764410ea5b76db0171903df => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:54.297405 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3ac57f-1c6f-430c-908c-d8765b9f2f80 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:54.366235 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d9ee2c-e4b6-49ec-aeee-a8fa87761d77 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:54.373646 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:54.414357 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615f4b5a-4439-4eee-887a-ab0aa3363361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:54.415242 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615f4b5a-4439-4eee-887a-ab0aa3363361 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:54.415705 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615f4b5a-4439-4eee-887a-ab0aa3363361 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:54.416107 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615f4b5a-4439-4eee-887a-ab0aa3363361 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:54.420874 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3ac57f-1c6f-430c-908c-d8765b9f2f80 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:54.468819 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:54.484735 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0ad7b1-8c2a-42d1-b26d-0a2f4edc7d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:54.485267 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ad7b1-8c2a-42d1-b26d-0a2f4edc7d38 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:54.485548 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ad7b1-8c2a-42d1-b26d-0a2f4edc7d38 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:54.485816 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ad7b1-8c2a-42d1-b26d-0a2f4edc7d38 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:54.486540 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0ad7b1-8c2a-42d1-b26d-0a2f4edc7d38 None admin] RBAC: Adding query filter params (group.id=519fd3c5007f452e8c09b26b1c49ccce, role.id=81991615ac0b46818426bf37015df930, scope.project.id=410027d6c764410ea5b76db0171903df) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:01:54.486841 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0ad7b1-8c2a-42d1-b26d-0a2f4edc7d38 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:54.488317 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0ad7b1-8c2a-42d1-b26d-0a2f4edc7d38 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:54.494950 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615f4b5a-4439-4eee-887a-ab0aa3363361 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:54.503004 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615f4b5a-4439-4eee-887a-ab0aa3363361 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:54.503191 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 161/316] 10.222.0.36 () {56 vars in 1376 bytes} [Wed Oct 12 00:01:54 2022] GET /identity/v3/role_assignments?group.id=519fd3c5007f452e8c09b26b1c49ccce&scope.project.id=410027d6c764410ea5b76db0171903df&role.id=81991615ac0b46818426bf37015df930 => generated 611 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:54.503191 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80cd7ae-d7af-4070-90d0-869944da0545 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:54.503191 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80cd7ae-d7af-4070-90d0-869944da0545 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:54.503191 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80cd7ae-d7af-4070-90d0-869944da0545 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:54.503191 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 162/317] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:01:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:01:54.503191 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80cd7ae-d7af-4070-90d0-869944da0545 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:54.639045 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 156/318] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Oct 12 00:01:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 12 00:01:54.640580 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126d0e96-d6e5-4d2e-a99f-32934d11d55f None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:54.640871 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126d0e96-d6e5-4d2e-a99f-32934d11d55f None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:54.641146 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126d0e96-d6e5-4d2e-a99f-32934d11d55f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:54.652675 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80cd7ae-d7af-4070-90d0-869944da0545 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:54.663318 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-126d0e96-d6e5-4d2e-a99f-32934d11d55f None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:01:54.664570 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-126d0e96-d6e5-4d2e-a99f-32934d11d55f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:01:54.701887 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283713b9-f3c0-48f7-82f2-8d0f72f7a4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:54.702535 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283713b9-f3c0-48f7-82f2-8d0f72f7a4f8 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:54.702817 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283713b9-f3c0-48f7-82f2-8d0f72f7a4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:54.703088 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283713b9-f3c0-48f7-82f2-8d0f72f7a4f8 None admin] PATH_INFO: `/v3/groups/519fd3c5007f452e8c09b26b1c49ccce` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:54.704234 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283713b9-f3c0-48f7-82f2-8d0f72f7a4f8 None admin] RBAC: Authorizing `identity:get_group(group_id=519fd3c5007f452e8c09b26b1c49ccce)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:54.705671 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283713b9-f3c0-48f7-82f2-8d0f72f7a4f8 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:54.707256 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 163/319] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Oct 12 00:01:54 2022] GET /identity/v3/groups/519fd3c5007f452e8c09b26b1c49ccce => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:54.708666 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283713b9-f3c0-48f7-82f2-8d0f72f7a4f8 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:54.718371 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:54.757071 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 157/320] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:01:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 00:01:54.757885 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126d0e96-d6e5-4d2e-a99f-32934d11d55f None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:54.773010 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:54.842761 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283713b9-f3c0-48f7-82f2-8d0f72f7a4f8 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:54.881784 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb20974-3395-449b-951a-9712f9b6139c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:54.882429 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb20974-3395-449b-951a-9712f9b6139c None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:54.882707 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb20974-3395-449b-951a-9712f9b6139c None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:54.882979 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb20974-3395-449b-951a-9712f9b6139c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:54.885601 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126d0e96-d6e5-4d2e-a99f-32934d11d55f None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:54.902598 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb20974-3395-449b-951a-9712f9b6139c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 00:01:54.909542 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-bfb20974-3395-449b-951a-9712f9b6139c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 12 00:01:54.909542 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:01:54.909542 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:01:54.909542 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:01:54.909542 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:01:54.909542 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:01:54.909542 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:01:54.909542 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:01:54.909542 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:01:54.909542 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:01:54.909542 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:01:54.909542 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:01:54.909542 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 00:01:54.909542 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 00:01:54.909542 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 00:01:54.909542 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 00:01:54.910468 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 00:01:54.910468 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 00:01:54.910468 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 00:01:54.910468 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 00:01:54.910468 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:01:54.910468 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:01:54.910468 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:01:54.910468 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:01:54.910468 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:01:54.910468 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:01:54.910468 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:01:54.910468 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:01:54.910468 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:01:54.910468 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:01:54.910468 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:01:54.910468 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:01:54.911770 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:01:54.911770 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:01:54.911770 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:01:54.911770 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:01:54.911770 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 00:01:54.911770 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 00:01:54.911770 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 00:01:54.911770 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 00:01:54.911770 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 00:01:54.911770 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 00:01:54.911770 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 12 00:01:54.911770 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:01:54.914709 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 164/321] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Oct 12 00:01:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:01:54.916013 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb20974-3395-449b-951a-9712f9b6139c None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:54.932305 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be65c648-a1e0-47a1-9a3c-1586b4988990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:54.933421 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be65c648-a1e0-47a1-9a3c-1586b4988990 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:54.934256 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be65c648-a1e0-47a1-9a3c-1586b4988990 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:54.934706 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be65c648-a1e0-47a1-9a3c-1586b4988990 None admin] PATH_INFO: `/v3/projects/25ba77a2f84b41cd99a774b93ada5e96` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:54.936404 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be65c648-a1e0-47a1-9a3c-1586b4988990 None admin] RBAC: Authorizing `identity:get_project(project_id=25ba77a2f84b41cd99a774b93ada5e96)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:54.938285 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be65c648-a1e0-47a1-9a3c-1586b4988990 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:54.940523 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 158/322] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Oct 12 00:01:54 2022] GET /identity/v3/projects/25ba77a2f84b41cd99a774b93ada5e96 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:54.942120 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be65c648-a1e0-47a1-9a3c-1586b4988990 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:54.991903 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:55.026799 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:55.035122 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb20974-3395-449b-951a-9712f9b6139c None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:55.050093 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be65c648-a1e0-47a1-9a3c-1586b4988990 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:55.070552 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b28410-2adf-4c59-9dd0-44443f6dd9b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:55.071150 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b28410-2adf-4c59-9dd0-44443f6dd9b8 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:55.071457 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b28410-2adf-4c59-9dd0-44443f6dd9b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:55.071727 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b28410-2adf-4c59-9dd0-44443f6dd9b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:55.072443 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b28410-2adf-4c59-9dd0-44443f6dd9b8 None admin] RBAC: Adding query filter params (group.id=519fd3c5007f452e8c09b26b1c49ccce, role.id=ef689f96442945e8a7bc8cbfee13d909, scope.project.id=410027d6c764410ea5b76db0171903df) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:01:55.072754 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b28410-2adf-4c59-9dd0-44443f6dd9b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:55.074193 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b28410-2adf-4c59-9dd0-44443f6dd9b8 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:55.079753 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 165/323] 10.222.0.36 () {56 vars in 1376 bytes} [Wed Oct 12 00:01:54 2022] GET /identity/v3/role_assignments?group.id=519fd3c5007f452e8c09b26b1c49ccce&scope.project.id=410027d6c764410ea5b76db0171903df&role.id=ef689f96442945e8a7bc8cbfee13d909 => generated 611 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:55.080512 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b28410-2adf-4c59-9dd0-44443f6dd9b8 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:55.083579 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da0622d-103f-48fc-9bcd-e2fde3dfea5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:55.084200 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da0622d-103f-48fc-9bcd-e2fde3dfea5e None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:55.084474 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da0622d-103f-48fc-9bcd-e2fde3dfea5e None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:55.084741 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da0622d-103f-48fc-9bcd-e2fde3dfea5e None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:55.085404 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da0622d-103f-48fc-9bcd-e2fde3dfea5e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:01:55.085702 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da0622d-103f-48fc-9bcd-e2fde3dfea5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:55.087110 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da0622d-103f-48fc-9bcd-e2fde3dfea5e None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:55.103448 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 159/324] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Oct 12 00:01:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:55.104328 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da0622d-103f-48fc-9bcd-e2fde3dfea5e None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:55.201077 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b28410-2adf-4c59-9dd0-44443f6dd9b8 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:55.219318 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da0622d-103f-48fc-9bcd-e2fde3dfea5e None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:55.232536 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:55.241624 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b78ce3-4c39-4a43-9533-3ea52c6ab8c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:55.242321 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b78ce3-4c39-4a43-9533-3ea52c6ab8c5 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:55.242592 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b78ce3-4c39-4a43-9533-3ea52c6ab8c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:55.242859 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b78ce3-4c39-4a43-9533-3ea52c6ab8c5 None admin] PATH_INFO: `/v3/groups/519fd3c5007f452e8c09b26b1c49ccce` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:55.244018 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b78ce3-4c39-4a43-9533-3ea52c6ab8c5 None admin] RBAC: Authorizing `identity:get_group(group_id=519fd3c5007f452e8c09b26b1c49ccce)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:55.245508 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b78ce3-4c39-4a43-9533-3ea52c6ab8c5 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:55.248563 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 166/325] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Oct 12 00:01:55 2022] GET /identity/v3/groups/519fd3c5007f452e8c09b26b1c49ccce => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:55.248563 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b78ce3-4c39-4a43-9533-3ea52c6ab8c5 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:55.265702 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f75693b-4a56-4feb-89f4-f886eb2a7b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:55.266325 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f75693b-4a56-4feb-89f4-f886eb2a7b81 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:55.266601 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f75693b-4a56-4feb-89f4-f886eb2a7b81 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:55.266870 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f75693b-4a56-4feb-89f4-f886eb2a7b81 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:55.278015 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:55.282583 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f75693b-4a56-4feb-89f4-f886eb2a7b81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 00:01:55.288420 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-5f75693b-4a56-4feb-89f4-f886eb2a7b81 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 00:01:55.288420 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:01:55.288420 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:01:55.288420 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:01:55.288420 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:01:55.288420 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:01:55.288420 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:01:55.288420 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:01:55.288420 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:01:55.288420 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:01:55.288420 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:01:55.288420 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:01:55.288420 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 00:01:55.288420 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 00:01:55.288420 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 00:01:55.288420 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 00:01:55.289710 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 00:01:55.289710 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 00:01:55.289710 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 00:01:55.289710 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 00:01:55.289710 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:01:55.289710 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:01:55.289710 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:01:55.289710 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:01:55.289710 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:01:55.289710 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:01:55.289710 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:01:55.289710 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:01:55.289710 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:01:55.289710 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:01:55.289710 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:01:55.289710 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:01:55.290495 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:01:55.290495 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:01:55.290495 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:01:55.290495 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:01:55.290495 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 00:01:55.290495 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 00:01:55.290495 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 00:01:55.290495 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 00:01:55.290495 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 00:01:55.290495 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 00:01:55.290495 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 00:01:55.290495 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:01:55.290495 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 160/326] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Oct 12 00:01:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:01:55.291170 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f75693b-4a56-4feb-89f4-f886eb2a7b81 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:55.374586 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b78ce3-4c39-4a43-9533-3ea52c6ab8c5 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:55.406430 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f75693b-4a56-4feb-89f4-f886eb2a7b81 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:55.417117 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d87c58d-3080-4d2b-b4e4-e5588b0fa74a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:55.417734 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d87c58d-3080-4d2b-b4e4-e5588b0fa74a None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:55.418015 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d87c58d-3080-4d2b-b4e4-e5588b0fa74a None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:55.418701 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d87c58d-3080-4d2b-b4e4-e5588b0fa74a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:55.443999 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceb6d29-bcc8-4f9d-8b91-ce0821473eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:55.443999 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceb6d29-bcc8-4f9d-8b91-ce0821473eae None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:55.443999 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceb6d29-bcc8-4f9d-8b91-ce0821473eae None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:55.443999 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceb6d29-bcc8-4f9d-8b91-ce0821473eae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:55.444647 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceb6d29-bcc8-4f9d-8b91-ce0821473eae None admin] RBAC: Adding query filter params (group.id=519fd3c5007f452e8c09b26b1c49ccce, role.id=ef689f96442945e8a7bc8cbfee13d909, scope.project.id=25ba77a2f84b41cd99a774b93ada5e96) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:01:55.444980 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceb6d29-bcc8-4f9d-8b91-ce0821473eae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:55.446442 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceb6d29-bcc8-4f9d-8b91-ce0821473eae None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:55.452409 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 161/327] 10.222.0.36 () {56 vars in 1376 bytes} [Wed Oct 12 00:01:55 2022] GET /identity/v3/role_assignments?group.id=519fd3c5007f452e8c09b26b1c49ccce&scope.project.id=25ba77a2f84b41cd99a774b93ada5e96&role.id=ef689f96442945e8a7bc8cbfee13d909 => generated 611 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:55.453204 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceb6d29-bcc8-4f9d-8b91-ce0821473eae None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:55.504715 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:55.517942 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d87c58d-3080-4d2b-b4e4-e5588b0fa74a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:55.519660 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d87c58d-3080-4d2b-b4e4-e5588b0fa74a None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:55.550773 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:55.608816 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceb6d29-bcc8-4f9d-8b91-ce0821473eae None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:55.648557 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c93b001-242d-4c87-b1f2-f14dfedabff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:55.649174 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c93b001-242d-4c87-b1f2-f14dfedabff5 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:55.649450 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c93b001-242d-4c87-b1f2-f14dfedabff5 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:55.649716 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c93b001-242d-4c87-b1f2-f14dfedabff5 None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:55.650391 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c93b001-242d-4c87-b1f2-f14dfedabff5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:01:55.650691 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c93b001-242d-4c87-b1f2-f14dfedabff5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:55.652216 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c93b001-242d-4c87-b1f2-f14dfedabff5 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:55.660134 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 167/328] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Oct 12 00:01:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 12 00:01:55.665224 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 162/329] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Oct 12 00:01:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:55.669070 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d87c58d-3080-4d2b-b4e4-e5588b0fa74a None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:55.671338 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c93b001-242d-4c87-b1f2-f14dfedabff5 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:55.784755 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:55.791728 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c93b001-242d-4c87-b1f2-f14dfedabff5 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:55.798605 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d87c58d-3080-4d2b-b4e4-e5588b0fa74a None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:55.820777 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:55.830806 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377eb078-a9b3-427a-8411-b72b8d3497d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:55.831465 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377eb078-a9b3-427a-8411-b72b8d3497d6 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:55.831741 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377eb078-a9b3-427a-8411-b72b8d3497d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:55.832970 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7e446a-e2df-4d55-a834-28389c169f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:55.833096 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377eb078-a9b3-427a-8411-b72b8d3497d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:55.833182 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7e446a-e2df-4d55-a834-28389c169f9d None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:55.833339 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7e446a-e2df-4d55-a834-28389c169f9d None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:55.833595 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7e446a-e2df-4d55-a834-28389c169f9d None admin] PATH_INFO: `/v3/projects/25ba77a2f84b41cd99a774b93ada5e96` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:55.835802 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7e446a-e2df-4d55-a834-28389c169f9d None admin] RBAC: Authorizing `identity:get_project(project_id=25ba77a2f84b41cd99a774b93ada5e96)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:55.836217 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7e446a-e2df-4d55-a834-28389c169f9d None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:55.838157 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 168/330] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Oct 12 00:01:55 2022] GET /identity/v3/projects/25ba77a2f84b41cd99a774b93ada5e96 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:55.844023 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7e446a-e2df-4d55-a834-28389c169f9d None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:55.903532 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377eb078-a9b3-427a-8411-b72b8d3497d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:55.904984 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377eb078-a9b3-427a-8411-b72b8d3497d6 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:55.945751 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7e446a-e2df-4d55-a834-28389c169f9d None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:55.979413 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dd4150-504e-4dae-8151-801b2f904fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:55.980043 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd4150-504e-4dae-8151-801b2f904fd8 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:55.980322 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd4150-504e-4dae-8151-801b2f904fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:55.980587 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd4150-504e-4dae-8151-801b2f904fd8 None admin] PATH_INFO: `/v3/groups/519fd3c5007f452e8c09b26b1c49ccce` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:55.981672 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dd4150-504e-4dae-8151-801b2f904fd8 None admin] RBAC: Authorizing `identity:get_group(group_id=519fd3c5007f452e8c09b26b1c49ccce)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:55.983090 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dd4150-504e-4dae-8151-801b2f904fd8 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:55.984839 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 169/331] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Oct 12 00:01:55 2022] GET /identity/v3/groups/519fd3c5007f452e8c09b26b1c49ccce => generated 226 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 00:01:55.990456 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dd4150-504e-4dae-8151-801b2f904fd8 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:56.003028 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 163/332] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Oct 12 00:01:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 12 00:01:56.004041 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377eb078-a9b3-427a-8411-b72b8d3497d6 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:56.022051 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:56.030743 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:56.090562 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dd4150-504e-4dae-8151-801b2f904fd8 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:56.101073 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377eb078-a9b3-427a-8411-b72b8d3497d6 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:56.123236 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb2b702-7878-4b0f-80b2-429f38ae71e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:56.123881 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb2b702-7878-4b0f-80b2-429f38ae71e8 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:56.124144 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb2b702-7878-4b0f-80b2-429f38ae71e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:56.124414 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb2b702-7878-4b0f-80b2-429f38ae71e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:56.125123 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb2b702-7878-4b0f-80b2-429f38ae71e8 None admin] RBAC: Adding query filter params (group.id=519fd3c5007f452e8c09b26b1c49ccce, role.id=81991615ac0b46818426bf37015df930, scope.project.id=25ba77a2f84b41cd99a774b93ada5e96) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:01:56.125424 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb2b702-7878-4b0f-80b2-429f38ae71e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:56.126854 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb2b702-7878-4b0f-80b2-429f38ae71e8 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:56.133008 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 170/333] 10.222.0.36 () {56 vars in 1376 bytes} [Wed Oct 12 00:01:55 2022] GET /identity/v3/role_assignments?group.id=519fd3c5007f452e8c09b26b1c49ccce&scope.project.id=25ba77a2f84b41cd99a774b93ada5e96&role.id=81991615ac0b46818426bf37015df930 => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 00:01:56.134440 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b2920a-e13e-4a1d-9b42-c470ee9e531e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:56.135321 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b2920a-e13e-4a1d-9b42-c470ee9e531e None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:56.135628 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b2920a-e13e-4a1d-9b42-c470ee9e531e None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:56.135899 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b2920a-e13e-4a1d-9b42-c470ee9e531e None admin] PATH_INFO: `/v3/projects/392dba9b727b479aa54a2746402ded73` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:56.137051 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b2920a-e13e-4a1d-9b42-c470ee9e531e None admin] RBAC: Authorizing `identity:get_project(project_id=392dba9b727b479aa54a2746402ded73)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:56.138520 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b2920a-e13e-4a1d-9b42-c470ee9e531e None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:56.140266 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 164/334] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Oct 12 00:01:56 2022] GET /identity/v3/projects/392dba9b727b479aa54a2746402ded73 => generated 344 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:56.146096 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb2b702-7878-4b0f-80b2-429f38ae71e8 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:56.243999 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:56.249795 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb2b702-7878-4b0f-80b2-429f38ae71e8 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:56.285599 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461a6de6-0cde-4dee-8526-b09692c3c916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:56.286240 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461a6de6-0cde-4dee-8526-b09692c3c916 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:56.286511 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461a6de6-0cde-4dee-8526-b09692c3c916 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:56.286776 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461a6de6-0cde-4dee-8526-b09692c3c916 None admin] PATH_INFO: `/v3/groups/f51f9cce250a4b209eabc1ea9630524f` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:56.287906 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461a6de6-0cde-4dee-8526-b09692c3c916 None admin] RBAC: Authorizing `identity:get_group(group_id=f51f9cce250a4b209eabc1ea9630524f)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:56.289353 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461a6de6-0cde-4dee-8526-b09692c3c916 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:56.290928 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 171/335] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Oct 12 00:01:56 2022] GET /identity/v3/groups/f51f9cce250a4b209eabc1ea9630524f => generated 229 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:56.296799 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b2920a-e13e-4a1d-9b42-c470ee9e531e None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:56.396706 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:56.403212 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b2920a-e13e-4a1d-9b42-c470ee9e531e None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:56.439886 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd3701c-26cf-4650-b478-19610481f3b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:56.440490 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd3701c-26cf-4650-b478-19610481f3b2 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:56.440764 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd3701c-26cf-4650-b478-19610481f3b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:56.441031 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd3701c-26cf-4650-b478-19610481f3b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:56.442304 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd3701c-26cf-4650-b478-19610481f3b2 None admin] RBAC: Adding query filter params (group.id=f51f9cce250a4b209eabc1ea9630524f, role.id=b2e41f0d7a304dc7af8b6a232049b6a2, scope.project.id=392dba9b727b479aa54a2746402ded73) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:01:56.442606 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd3701c-26cf-4650-b478-19610481f3b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:56.444126 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd3701c-26cf-4650-b478-19610481f3b2 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:56.450135 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 165/336] 10.222.0.36 () {56 vars in 1376 bytes} [Wed Oct 12 00:01:56 2022] GET /identity/v3/role_assignments?group.id=f51f9cce250a4b209eabc1ea9630524f&scope.project.id=392dba9b727b479aa54a2746402ded73&role.id=b2e41f0d7a304dc7af8b6a232049b6a2 => generated 611 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:58.919110 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8afb23f-ab76-4121-a451-e2954df23dcc None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:58.919693 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8afb23f-ab76-4121-a451-e2954df23dcc None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:58.919693 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8afb23f-ab76-4121-a451-e2954df23dcc None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:58.921295 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 172/337] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:01:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:01:58.926334 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9f923-3a2f-448c-9704-8516aa60768b None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:58.926761 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9f923-3a2f-448c-9704-8516aa60768b None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:58.926940 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9f923-3a2f-448c-9704-8516aa60768b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:58.949740 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-a3c9f923-3a2f-448c-9704-8516aa60768b None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:01:58.950933 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-a3c9f923-3a2f-448c-9704-8516aa60768b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:01:59.054064 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 166/338] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:01:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 00:01:59.157888 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b6cccc-1cb8-454e-9e86-b7e8a9e8ccde None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:59.158231 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b6cccc-1cb8-454e-9e86-b7e8a9e8ccde None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:59.158429 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b6cccc-1cb8-454e-9e86-b7e8a9e8ccde None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:59.159932 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 173/339] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:01:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:01:59.165676 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc620ceb-3d8e-4d16-95c4-66e62b08cdd4 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:59.165963 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc620ceb-3d8e-4d16-95c4-66e62b08cdd4 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:59.166235 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc620ceb-3d8e-4d16-95c4-66e62b08cdd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:59.192346 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-dc620ceb-3d8e-4d16-95c4-66e62b08cdd4 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:01:59.193450 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-dc620ceb-3d8e-4d16-95c4-66e62b08cdd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:01:59.291529 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 167/340] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:01:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 00:01:59.382338 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b6cccc-1cb8-454e-9e86-b7e8a9e8ccde None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:59.404360 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e619c90-0ced-481f-8df5-96c196fed41c None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:59.404666 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e619c90-0ced-481f-8df5-96c196fed41c None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:59.404932 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e619c90-0ced-481f-8df5-96c196fed41c None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:59.406559 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 168/341] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:01:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:01:59.412440 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23a4e4d-fa7e-405b-8c3e-ad2791126ed7 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:59.412719 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23a4e4d-fa7e-405b-8c3e-ad2791126ed7 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:59.412992 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23a4e4d-fa7e-405b-8c3e-ad2791126ed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:59.437221 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-c23a4e4d-fa7e-405b-8c3e-ad2791126ed7 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:01:59.438625 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-c23a4e4d-fa7e-405b-8c3e-ad2791126ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:01:59.462584 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:59.548600 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b6cccc-1cb8-454e-9e86-b7e8a9e8ccde None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:59.583183 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 169/342] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:01:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 12 00:01:59.603251 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176bcac5-d9f3-4d2f-909b-8a8b6334dc45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:59.604217 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176bcac5-d9f3-4d2f-909b-8a8b6334dc45 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:59.604694 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176bcac5-d9f3-4d2f-909b-8a8b6334dc45 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:59.605170 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176bcac5-d9f3-4d2f-909b-8a8b6334dc45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:59.620861 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23a4e4d-fa7e-405b-8c3e-ad2791126ed7 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:59.634109 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176bcac5-d9f3-4d2f-909b-8a8b6334dc45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:59.638093 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176bcac5-d9f3-4d2f-909b-8a8b6334dc45 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:59.671396 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-176bcac5-d9f3-4d2f-909b-8a8b6334dc45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:01:59.671396 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:01:59.671396 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:01:59.671396 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:01:59.671396 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:01:59.671396 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:01:59.671396 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:01:59.671396 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:01:59.671396 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:01:59.671396 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:01:59.671396 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:01:59.671396 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:01:59.671396 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 00:01:59.671396 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 00:01:59.671396 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 00:01:59.671396 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 00:01:59.672422 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:01:59.672422 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:01:59.672422 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:01:59.672422 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:01:59.672422 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:01:59.672422 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:01:59.672422 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:01:59.672422 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:01:59.672422 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:01:59.672422 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:01:59.672422 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:01:59.672422 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:01:59.672422 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:01:59.672422 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:01:59.672422 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:01:59.672422 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:01:59.672422 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 00:01:59.673267 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 00:01:59.673267 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:01:59.673267 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:01:59.683549 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 174/343] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:01:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:01:59.684788 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176bcac5-d9f3-4d2f-909b-8a8b6334dc45 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:59.730159 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:59.843950 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23a4e4d-fa7e-405b-8c3e-ad2791126ed7 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:59.861021 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176bcac5-d9f3-4d2f-909b-8a8b6334dc45 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:01:59.901294 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d6532d-13c9-4fbf-b67e-2af3923b41c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:59.901959 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d6532d-13c9-4fbf-b67e-2af3923b41c8 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:59.902311 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d6532d-13c9-4fbf-b67e-2af3923b41c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:59.902577 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d6532d-13c9-4fbf-b67e-2af3923b41c8 None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:59.903213 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d6532d-13c9-4fbf-b67e-2af3923b41c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:01:59.903537 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d6532d-13c9-4fbf-b67e-2af3923b41c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:59.904863 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb37c4b-0193-4ea9-8f3d-a9c2d82f36b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:01:59.904983 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d6532d-13c9-4fbf-b67e-2af3923b41c8 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:59.905817 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb37c4b-0193-4ea9-8f3d-a9c2d82f36b8 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:01:59.906294 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb37c4b-0193-4ea9-8f3d-a9c2d82f36b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:01:59.906916 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb37c4b-0193-4ea9-8f3d-a9c2d82f36b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:01:59.918322 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 175/344] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Oct 12 00:01:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:01:59.919195 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d6532d-13c9-4fbf-b67e-2af3923b41c8 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:01:59.920088 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb37c4b-0193-4ea9-8f3d-a9c2d82f36b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:01:59.922006 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb37c4b-0193-4ea9-8f3d-a9c2d82f36b8 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:01:59.934208 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:01:59.945571 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-5bb37c4b-0193-4ea9-8f3d-a9c2d82f36b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:01:59.945571 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:01:59.945571 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:01:59.945571 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:01:59.945571 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:01:59.945571 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:01:59.945571 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:01:59.945571 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:01:59.945571 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:01:59.945571 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:01:59.945571 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:01:59.945571 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:01:59.945571 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 00:01:59.945571 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 00:01:59.945571 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 00:01:59.945571 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 00:01:59.946438 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:01:59.946438 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:01:59.946438 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:01:59.946438 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:01:59.946438 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:01:59.946438 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:01:59.946438 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:01:59.946438 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:01:59.946438 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:01:59.946438 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:01:59.946438 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:01:59.946438 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:01:59.946438 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:01:59.946438 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:01:59.946438 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:01:59.946438 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:01:59.946438 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 00:01:59.947302 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 00:01:59.947302 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:01:59.947302 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:01:59.949831 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 170/345] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:01:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:01:59.951546 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb37c4b-0193-4ea9-8f3d-a9c2d82f36b8 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:00.009851 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:00.036115 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d6532d-13c9-4fbf-b67e-2af3923b41c8 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:00.080440 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9a9e1f-42e9-42e3-be31-a7eedf05a226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:00.081126 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9a9e1f-42e9-42e3-be31-a7eedf05a226 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:00.081689 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9a9e1f-42e9-42e3-be31-a7eedf05a226 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:00.082007 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9a9e1f-42e9-42e3-be31-a7eedf05a226 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:00.093333 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb37c4b-0193-4ea9-8f3d-a9c2d82f36b8 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:00.102250 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9a9e1f-42e9-42e3-be31-a7eedf05a226 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:00.104790 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9a9e1f-42e9-42e3-be31-a7eedf05a226 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:00.125972 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-4d9a9e1f-42e9-42e3-be31-a7eedf05a226 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:00.125972 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:00.125972 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:00.125972 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:00.125972 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:00.125972 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:00.125972 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:00.125972 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:00.125972 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:00.125972 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:00.125972 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:00.125972 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:00.125972 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 00:02:00.125972 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 00:02:00.125972 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 00:02:00.125972 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 00:02:00.126883 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:00.126883 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:00.126883 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:00.126883 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:00.126883 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:00.126883 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:00.126883 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:00.126883 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:00.126883 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:00.126883 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:00.126883 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:00.126883 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:00.126883 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:00.126883 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:00.126883 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:00.126883 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:00.126883 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 00:02:00.127752 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 00:02:00.127752 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:00.127752 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:00.134124 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 176/346] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:01:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:00.134124 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9a9e1f-42e9-42e3-be31-a7eedf05a226 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:00.138856 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d7af84-d6dd-49d9-9068-05d200a6fdc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:00.139501 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d7af84-d6dd-49d9-9068-05d200a6fdc4 None admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:00.139777 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d7af84-d6dd-49d9-9068-05d200a6fdc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:00.140041 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d7af84-d6dd-49d9-9068-05d200a6fdc4 None admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:00.140749 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d7af84-d6dd-49d9-9068-05d200a6fdc4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:00.142311 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d7af84-d6dd-49d9-9068-05d200a6fdc4 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:00.238174 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 171/347] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Oct 12 00:01:59 2022] POST /identity/v3/users => generated 246 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:02:00.239618 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d7af84-d6dd-49d9-9068-05d200a6fdc4 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:00.254179 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9a9e1f-42e9-42e3-be31-a7eedf05a226 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:00.291135 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:00.302870 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290b00d8-4c8e-4e99-a813-7455127d93c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:00.304108 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290b00d8-4c8e-4e99-a813-7455127d93c1 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:00.304545 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290b00d8-4c8e-4e99-a813-7455127d93c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:00.305005 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290b00d8-4c8e-4e99-a813-7455127d93c1 None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:00.306867 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:00.307699 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290b00d8-4c8e-4e99-a813-7455127d93c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:00.308561 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290b00d8-4c8e-4e99-a813-7455127d93c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:00.310156 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290b00d8-4c8e-4e99-a813-7455127d93c1 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:00.327399 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 177/348] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Oct 12 00:02:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:00.333856 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358843a0-ad21-44bf-8c91-1c8c12e9d0ec None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:00.333856 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358843a0-ad21-44bf-8c91-1c8c12e9d0ec None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:00.333856 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358843a0-ad21-44bf-8c91-1c8c12e9d0ec None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:00.333856 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 178/349] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:02:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:02:00.333856 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358843a0-ad21-44bf-8c91-1c8c12e9d0ec None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:00.358888 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d7af84-d6dd-49d9-9068-05d200a6fdc4 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:00.399884 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e9fcb6-2363-4622-9dbc-4ddc5182b4a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:00.400635 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e9fcb6-2363-4622-9dbc-4ddc5182b4a8 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:00.400989 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e9fcb6-2363-4622-9dbc-4ddc5182b4a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:00.401597 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e9fcb6-2363-4622-9dbc-4ddc5182b4a8 None admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:00.402042 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e9fcb6-2363-4622-9dbc-4ddc5182b4a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:00.402484 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e9fcb6-2363-4622-9dbc-4ddc5182b4a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:00.404151 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e9fcb6-2363-4622-9dbc-4ddc5182b4a8 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:00.421636 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 172/350] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Oct 12 00:02:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:00.421636 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9c8171-5e31-4aae-9f74-ee58b2b17643 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:00.421636 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9c8171-5e31-4aae-9f74-ee58b2b17643 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:00.421636 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9c8171-5e31-4aae-9f74-ee58b2b17643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:00.441823 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-5e9c8171-5e31-4aae-9f74-ee58b2b17643 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:02:00.443093 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-5e9c8171-5e31-4aae-9f74-ee58b2b17643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:02:00.469191 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358843a0-ad21-44bf-8c91-1c8c12e9d0ec None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:00.507289 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a38e7fe-cfd3-4867-96ce-9bc555fa9fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:00.507955 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a38e7fe-cfd3-4867-96ce-9bc555fa9fe3 None admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:00.508232 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a38e7fe-cfd3-4867-96ce-9bc555fa9fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:00.509113 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a38e7fe-cfd3-4867-96ce-9bc555fa9fe3 None admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:00.511408 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a38e7fe-cfd3-4867-96ce-9bc555fa9fe3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:00.511952 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a38e7fe-cfd3-4867-96ce-9bc555fa9fe3 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:00.522101 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 173/351] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:02:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 00:02:00.523293 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9c8171-5e31-4aae-9f74-ee58b2b17643 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:00.579804 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:00.593154 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 179/352] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Oct 12 00:02:00 2022] POST /identity/v3/users => generated 248 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:02:00.640694 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c0433a-4970-4293-b00c-fc99af91c342 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:00.641044 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c0433a-4970-4293-b00c-fc99af91c342 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:00.641322 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c0433a-4970-4293-b00c-fc99af91c342 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:00.643264 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 180/353] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:02:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 00:02:00.649291 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f499dc89-7ad6-46c6-9aac-0468142fc8ef None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:00.649467 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f499dc89-7ad6-46c6-9aac-0468142fc8ef None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:00.649691 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f499dc89-7ad6-46c6-9aac-0468142fc8ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:00.662201 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9c8171-5e31-4aae-9f74-ee58b2b17643 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:00.684194 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-f499dc89-7ad6-46c6-9aac-0468142fc8ef None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:02:00.685943 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-f499dc89-7ad6-46c6-9aac-0468142fc8ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:02:00.733611 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bd9deb-5176-4b6c-8c1d-77c5d38f9332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:00.735450 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bd9deb-5176-4b6c-8c1d-77c5d38f9332 None admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:00.736702 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bd9deb-5176-4b6c-8c1d-77c5d38f9332 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:00.737214 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bd9deb-5176-4b6c-8c1d-77c5d38f9332 None admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:00.738637 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bd9deb-5176-4b6c-8c1d-77c5d38f9332 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:00.740589 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bd9deb-5176-4b6c-8c1d-77c5d38f9332 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:00.764506 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:00.794773 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 181/354] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:02:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 00:02:00.834060 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f499dc89-7ad6-46c6-9aac-0468142fc8ef None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:00.871357 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 174/355] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Oct 12 00:02:00 2022] POST /identity/v3/users => generated 248 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:02:00.955264 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f499dc89-7ad6-46c6-9aac-0468142fc8ef None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:01.004163 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed5f19b-bc18-4c1a-a1c9-99940b6ae5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:01.004823 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed5f19b-bc18-4c1a-a1c9-99940b6ae5e9 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:01.005098 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed5f19b-bc18-4c1a-a1c9-99940b6ae5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:01.005366 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed5f19b-bc18-4c1a-a1c9-99940b6ae5e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:01.017430 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed5f19b-bc18-4c1a-a1c9-99940b6ae5e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:01.018973 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed5f19b-bc18-4c1a-a1c9-99940b6ae5e9 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:01.034904 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-1ed5f19b-bc18-4c1a-a1c9-99940b6ae5e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:01.034904 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:01.034904 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:01.034904 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:01.034904 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:01.034904 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:01.034904 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:01.034904 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:01.034904 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:01.034904 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:01.034904 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:01.034904 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:01.034904 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 00:02:01.034904 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 00:02:01.034904 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 00:02:01.034904 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 00:02:01.036396 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:01.036396 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:01.036396 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:01.036396 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:01.036396 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:01.036396 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:01.036396 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:01.036396 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:01.036396 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:01.036396 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:01.036396 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:01.036396 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:01.036396 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:01.036396 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:01.036396 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:01.036396 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:01.036396 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 00:02:01.037877 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 00:02:01.037877 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:01.037877 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:01.037877 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 182/356] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:02:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 00:02:01.044497 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bd9deb-5176-4b6c-8c1d-77c5d38f9332 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:01.066695 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed5f19b-bc18-4c1a-a1c9-99940b6ae5e9 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:01.099832 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:01.182049 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed5f19b-bc18-4c1a-a1c9-99940b6ae5e9 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:01.196870 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bd9deb-5176-4b6c-8c1d-77c5d38f9332 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:01.199425 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:01.230485 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279a01f8-a4a5-487c-90c1-a55f4ecb3988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:01.231658 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279a01f8-a4a5-487c-90c1-a55f4ecb3988 None admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:01.232160 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279a01f8-a4a5-487c-90c1-a55f4ecb3988 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:01.232629 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279a01f8-a4a5-487c-90c1-a55f4ecb3988 None admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:01.233522 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279a01f8-a4a5-487c-90c1-a55f4ecb3988 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:01.235116 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279a01f8-a4a5-487c-90c1-a55f4ecb3988 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:01.248485 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dec758c-c9d3-41af-b965-e2565943b4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:01.249103 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dec758c-c9d3-41af-b965-e2565943b4ec None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:01.249376 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dec758c-c9d3-41af-b965-e2565943b4ec None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:01.249641 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dec758c-c9d3-41af-b965-e2565943b4ec None admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:01.250282 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dec758c-c9d3-41af-b965-e2565943b4ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:01.250578 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dec758c-c9d3-41af-b965-e2565943b4ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:01.252247 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dec758c-c9d3-41af-b965-e2565943b4ec None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:01.265804 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 175/357] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Oct 12 00:02:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:01.272070 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dec758c-c9d3-41af-b965-e2565943b4ec None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:01.276079 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.sql.core [None req-279a01f8-a4a5-487c-90c1-a55f4ecb3988 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 12 00:02:01.276079 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 12 00:02:01.276079 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [parameters: {'id': '796d79c5bacc4ad6b5a45a3176bc0454', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 12 00:02:01.276079 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81747) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 12 00:02:01.284006 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-279a01f8-a4a5-487c-90c1-a55f4ecb3988 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 12 00:02:01.284006 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:01.284006 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 12 00:02:01.284006 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 12 00:02:01.284006 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 12 00:02:01.284006 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 12 00:02:01.284006 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 12 00:02:01.284006 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application result = self._query(query) Oct 12 00:02:01.284006 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 12 00:02:01.284006 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application conn.query(q) Oct 12 00:02:01.284006 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 12 00:02:01.284006 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 12 00:02:01.284006 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 12 00:02:01.284006 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application result.read() Oct 12 00:02:01.284006 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 12 00:02:01.284910 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 12 00:02:01.284910 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 12 00:02:01.284910 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 12 00:02:01.284910 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 12 00:02:01.284910 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 12 00:02:01.284910 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 12 00:02:01.284910 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 12 00:02:01.284910 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 12 00:02:01.284910 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:01.284910 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 12 00:02:01.284910 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:01.284910 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:01.284910 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 12 00:02:01.284910 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 12 00:02:01.284910 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Oct 12 00:02:01.284910 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return ref.to_dict() Oct 12 00:02:01.284910 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 12 00:02:01.284910 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application next(self.gen) Oct 12 00:02:01.286365 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 12 00:02:01.286365 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application yield resource Oct 12 00:02:01.286365 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 12 00:02:01.286365 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application next(self.gen) Oct 12 00:02:01.286365 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 12 00:02:01.286365 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application self.session.rollback() Oct 12 00:02:01.286365 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 12 00:02:01.286365 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application self.force_reraise() Oct 12 00:02:01.286365 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 12 00:02:01.286365 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise self.value Oct 12 00:02:01.286365 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 12 00:02:01.286365 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 12 00:02:01.286365 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 12 00:02:01.286365 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application session.commit() Oct 12 00:02:01.286365 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 12 00:02:01.286365 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 12 00:02:01.286365 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 12 00:02:01.287225 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application self._prepare_impl() Oct 12 00:02:01.287225 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 12 00:02:01.287225 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application self.session.flush() Oct 12 00:02:01.287225 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 12 00:02:01.287225 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application self._flush(objects) Oct 12 00:02:01.287225 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 12 00:02:01.287225 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 12 00:02:01.287225 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 12 00:02:01.287225 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application compat.raise_( Oct 12 00:02:01.287225 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 12 00:02:01.287225 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception Oct 12 00:02:01.287225 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 12 00:02:01.287225 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application flush_context.execute() Oct 12 00:02:01.287225 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 12 00:02:01.287225 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rec.execute(self) Oct 12 00:02:01.287225 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 12 00:02:01.287225 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 12 00:02:01.289033 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 12 00:02:01.289033 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 12 00:02:01.289033 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 12 00:02:01.289033 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 12 00:02:01.289033 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 12 00:02:01.289033 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 12 00:02:01.289033 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 12 00:02:01.289033 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 12 00:02:01.289033 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 12 00:02:01.289033 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 12 00:02:01.289033 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 12 00:02:01.289033 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 12 00:02:01.289033 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 12 00:02:01.289033 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 12 00:02:01.289033 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 12 00:02:01.289033 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception Oct 12 00:02:01.290875 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 12 00:02:01.290875 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 12 00:02:01.290875 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 12 00:02:01.290875 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 12 00:02:01.290875 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 12 00:02:01.290875 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application result = self._query(query) Oct 12 00:02:01.290875 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 12 00:02:01.290875 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application conn.query(q) Oct 12 00:02:01.290875 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 12 00:02:01.290875 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 12 00:02:01.290875 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 12 00:02:01.290875 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application result.read() Oct 12 00:02:01.290875 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 12 00:02:01.290875 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 12 00:02:01.290875 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 12 00:02:01.290875 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 12 00:02:01.292274 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 12 00:02:01.292274 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 12 00:02:01.292274 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 12 00:02:01.292274 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 12 00:02:01.292274 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 12 00:02:01.292274 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 12 00:02:01.292274 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [parameters: {'id': '796d79c5bacc4ad6b5a45a3176bc0454', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 12 00:02:01.292274 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 12 00:02:01.292274 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:01.292274 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 00:02:01.292274 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:01.292274 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:01.292274 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:01.292274 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:01.292274 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:01.292274 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:01.294110 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:01.294110 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:01.294110 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:01.294110 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:01.294110 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:01.294110 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:01.294110 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Oct 12 00:02:01.294110 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Oct 12 00:02:01.294110 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:01.294110 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:01.294110 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Oct 12 00:02:01.294110 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Oct 12 00:02:01.294110 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 12 00:02:01.294110 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 12 00:02:01.294110 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 12 00:02:01.294110 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:01.297730 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 183/358] 10.222.0.36 () {60 vars in 1131 bytes} [Wed Oct 12 00:02:01 2022] POST /identity/v3/roles => generated 144 bytes in 231 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 12 00:02:01.304622 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279a01f8-a4a5-487c-90c1-a55f4ecb3988 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:01.415858 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dec758c-c9d3-41af-b965-e2565943b4ec None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:01.451554 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64200226-9790-483d-977e-035f0f612278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:01.452321 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64200226-9790-483d-977e-035f0f612278 None admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:01.452708 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64200226-9790-483d-977e-035f0f612278 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:01.453058 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64200226-9790-483d-977e-035f0f612278 None admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:01.453808 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64200226-9790-483d-977e-035f0f612278 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:01.455454 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64200226-9790-483d-977e-035f0f612278 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:01.481900 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:01.492619 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279a01f8-a4a5-487c-90c1-a55f4ecb3988 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:01.528439 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d5d3ed-2dc7-4e17-9f0d-02e6398c0993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:01.529453 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d5d3ed-2dc7-4e17-9f0d-02e6398c0993 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:01.529936 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d5d3ed-2dc7-4e17-9f0d-02e6398c0993 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:01.530433 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d5d3ed-2dc7-4e17-9f0d-02e6398c0993 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:01.549653 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 176/359] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Oct 12 00:02:01 2022] POST /identity/v3/users => generated 249 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 00:02:01.554259 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d5d3ed-2dc7-4e17-9f0d-02e6398c0993 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 00:02:01.560095 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-e3d5d3ed-2dc7-4e17-9f0d-02e6398c0993 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 12 00:02:01.560095 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:01.560095 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:01.560095 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:01.560095 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:01.560095 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:01.560095 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:01.560095 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:01.560095 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:01.560095 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:01.560095 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:01.560095 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:01.560095 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 00:02:01.560095 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 00:02:01.560095 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 00:02:01.560095 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 00:02:01.561013 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 00:02:01.561013 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 00:02:01.561013 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 00:02:01.561013 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 00:02:01.561013 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:01.561013 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:01.561013 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:01.561013 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:01.561013 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:01.561013 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:01.561013 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:01.561013 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:01.561013 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:01.561013 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:01.561013 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:01.561013 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:01.561796 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:01.561796 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:01.561796 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:01.561796 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:01.561796 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 00:02:01.561796 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 00:02:01.561796 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 00:02:01.561796 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 00:02:01.561796 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 00:02:01.561796 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 00:02:01.561796 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 12 00:02:01.561796 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:01.564521 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 184/360] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Oct 12 00:02:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 00:02:01.571284 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64200226-9790-483d-977e-035f0f612278 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:01.593871 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:01.696111 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64200226-9790-483d-977e-035f0f612278 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:01.745747 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acae5af6-3b98-43f9-bd7d-d715d1ce6861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:01.746385 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae5af6-3b98-43f9-bd7d-d715d1ce6861 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:01.746662 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae5af6-3b98-43f9-bd7d-d715d1ce6861 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:01.746929 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae5af6-3b98-43f9-bd7d-d715d1ce6861 None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:01.747631 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acae5af6-3b98-43f9-bd7d-d715d1ce6861 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:01.747932 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acae5af6-3b98-43f9-bd7d-d715d1ce6861 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:01.749344 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acae5af6-3b98-43f9-bd7d-d715d1ce6861 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:01.760326 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 177/361] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Oct 12 00:02:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:02.938048 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2483adab-e8db-4098-acc6-014183a43827 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:02.939105 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2483adab-e8db-4098-acc6-014183a43827 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:02.939640 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2483adab-e8db-4098-acc6-014183a43827 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:02.941470 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 185/362] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:02:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:02:02.947009 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c6ff7f-e603-4ca9-89a6-5b3d8e76bc3b None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:02.947847 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c6ff7f-e603-4ca9-89a6-5b3d8e76bc3b None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:02.948354 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c6ff7f-e603-4ca9-89a6-5b3d8e76bc3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:02.971251 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-80c6ff7f-e603-4ca9-89a6-5b3d8e76bc3b None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:02:02.973701 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-80c6ff7f-e603-4ca9-89a6-5b3d8e76bc3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:02:03.069514 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 178/363] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:02:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 12 00:02:03.201644 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ad9428-cac1-478d-a13a-ce5187539de2 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:03.202252 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ad9428-cac1-478d-a13a-ce5187539de2 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:03.202753 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ad9428-cac1-478d-a13a-ce5187539de2 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:03.204586 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 186/364] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:02:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:02:03.209990 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e45b5d-524a-4e7a-b72d-ad75c6322915 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:03.210514 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e45b5d-524a-4e7a-b72d-ad75c6322915 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:03.211013 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e45b5d-524a-4e7a-b72d-ad75c6322915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:03.230910 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-71e45b5d-524a-4e7a-b72d-ad75c6322915 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:02:03.232439 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-71e45b5d-524a-4e7a-b72d-ad75c6322915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:02:03.240289 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:03.321281 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 179/365] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:02:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 00:02:03.460901 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ad9428-cac1-478d-a13a-ce5187539de2 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:03.588582 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ad9428-cac1-478d-a13a-ce5187539de2 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:03.640061 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaad31c-5e70-4796-9779-501f270a1c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:03.640705 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaad31c-5e70-4796-9779-501f270a1c3c None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:03.640981 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaad31c-5e70-4796-9779-501f270a1c3c None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:03.641251 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaad31c-5e70-4796-9779-501f270a1c3c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:03.655961 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaad31c-5e70-4796-9779-501f270a1c3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 00:02:03.660780 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-dbaad31c-5e70-4796-9779-501f270a1c3c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 12 00:02:03.660780 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:03.660780 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:03.660780 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:03.660780 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:03.660780 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:03.660780 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:03.660780 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:03.660780 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:03.660780 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:03.660780 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:03.660780 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:03.660780 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 00:02:03.660780 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 00:02:03.660780 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 00:02:03.660780 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 00:02:03.661775 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 00:02:03.661775 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 00:02:03.661775 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 00:02:03.661775 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 00:02:03.661775 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:03.661775 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:03.661775 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:03.661775 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:03.661775 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:03.661775 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:03.661775 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:03.661775 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:03.661775 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:03.661775 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:03.661775 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:03.661775 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:03.662606 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:03.662606 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:03.662606 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:03.662606 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:03.662606 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 00:02:03.662606 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 00:02:03.662606 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 00:02:03.662606 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 00:02:03.662606 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 00:02:03.662606 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 00:02:03.662606 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 12 00:02:03.662606 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:03.662606 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 187/366] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Oct 12 00:02:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:03.669739 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e45b5d-524a-4e7a-b72d-ad75c6322915 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:03.679140 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaad31c-5e70-4796-9779-501f270a1c3c None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:03.702876 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:03.833271 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e45b5d-524a-4e7a-b72d-ad75c6322915 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:03.869112 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252ae356-4692-4c0d-b19e-aeb6789175b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:03.869712 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252ae356-4692-4c0d-b19e-aeb6789175b8 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:03.869992 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252ae356-4692-4c0d-b19e-aeb6789175b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:03.870267 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252ae356-4692-4c0d-b19e-aeb6789175b8 None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:03.870930 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252ae356-4692-4c0d-b19e-aeb6789175b8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:03.871232 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252ae356-4692-4c0d-b19e-aeb6789175b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:03.872662 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252ae356-4692-4c0d-b19e-aeb6789175b8 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:03.882235 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 180/367] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Oct 12 00:02:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:03.885628 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaad31c-5e70-4796-9779-501f270a1c3c None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:03.886696 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6790a89d-188b-4e20-be13-20b728051585 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:03.886981 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6790a89d-188b-4e20-be13-20b728051585 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:03.887255 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6790a89d-188b-4e20-be13-20b728051585 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:03.892592 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 181/368] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:02:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 00:02:03.893384 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6790a89d-188b-4e20-be13-20b728051585 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:03.896250 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:03.923877 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d74051-e0cb-42ca-87fe-0c8245d885cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:03.924808 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d74051-e0cb-42ca-87fe-0c8245d885cc None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:03.925197 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d74051-e0cb-42ca-87fe-0c8245d885cc None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:03.926461 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d74051-e0cb-42ca-87fe-0c8245d885cc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:03.940399 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d74051-e0cb-42ca-87fe-0c8245d885cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 00:02:03.946435 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-a3d74051-e0cb-42ca-87fe-0c8245d885cc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 12 00:02:03.946435 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:03.946435 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:03.946435 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:03.946435 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:03.946435 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:03.946435 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:03.946435 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:03.946435 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:03.946435 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:03.946435 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:03.946435 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:03.946435 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 00:02:03.946435 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 00:02:03.946435 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 00:02:03.946435 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 00:02:03.948380 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 00:02:03.948380 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 00:02:03.948380 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 00:02:03.948380 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 00:02:03.948380 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:03.948380 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:03.948380 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:03.948380 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:03.948380 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:03.948380 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:03.948380 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:03.948380 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:03.948380 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:03.948380 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:03.948380 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:03.948380 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:03.950257 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:03.950257 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:03.950257 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:03.950257 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:03.950257 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 00:02:03.950257 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 00:02:03.950257 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 00:02:03.950257 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 00:02:03.950257 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 00:02:03.950257 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 00:02:03.950257 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 12 00:02:03.950257 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:03.950257 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 188/369] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Oct 12 00:02:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:03.954843 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c2559-3270-48dc-90a9-3e00f4f658c3 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:03.955029 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c2559-3270-48dc-90a9-3e00f4f658c3 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:03.959521 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c2559-3270-48dc-90a9-3e00f4f658c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:03.978409 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-b79c2559-3270-48dc-90a9-3e00f4f658c3 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:02:03.979822 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-b79c2559-3270-48dc-90a9-3e00f4f658c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:02:04.023726 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6790a89d-188b-4e20-be13-20b728051585 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:04.060778 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:04.065492 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee812f8-5d40-42b4-a574-74fbdceaafd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:04.066165 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee812f8-5d40-42b4-a574-74fbdceaafd0 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:04.066446 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee812f8-5d40-42b4-a574-74fbdceaafd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:04.066719 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee812f8-5d40-42b4-a574-74fbdceaafd0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:04.067471 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 189/370] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:02:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 00:02:04.076477 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79c2559-3270-48dc-90a9-3e00f4f658c3 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:04.082228 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee812f8-5d40-42b4-a574-74fbdceaafd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:04.082228 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee812f8-5d40-42b4-a574-74fbdceaafd0 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:04.098856 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-fee812f8-5d40-42b4-a574-74fbdceaafd0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:04.098856 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:04.098856 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:04.098856 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:04.098856 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:04.098856 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:04.098856 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:04.098856 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:04.098856 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:04.098856 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:04.098856 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:04.098856 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:04.098856 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 00:02:04.098856 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 00:02:04.098856 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 00:02:04.098856 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 00:02:04.100173 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:04.100173 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:04.100173 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:04.100173 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:04.100173 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:04.100173 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:04.100173 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:04.100173 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:04.100173 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:04.100173 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:04.100173 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:04.100173 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:04.100173 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:04.100173 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:04.100173 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:04.100173 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:04.100173 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 00:02:04.101039 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 00:02:04.101039 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:04.101039 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:04.103206 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 182/371] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:02:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:04.108205 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee812f8-5d40-42b4-a574-74fbdceaafd0 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:04.172321 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:04.226711 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee812f8-5d40-42b4-a574-74fbdceaafd0 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:04.272515 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdece14-b6cc-475b-b1c7-c22a8bdea6f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:04.273471 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdece14-b6cc-475b-b1c7-c22a8bdea6f4 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:04.273923 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdece14-b6cc-475b-b1c7-c22a8bdea6f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:04.274357 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdece14-b6cc-475b-b1c7-c22a8bdea6f4 None admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:04.275456 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdece14-b6cc-475b-b1c7-c22a8bdea6f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:04.276184 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdece14-b6cc-475b-b1c7-c22a8bdea6f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:04.278031 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdece14-b6cc-475b-b1c7-c22a8bdea6f4 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:04.295359 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 183/372] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Oct 12 00:02:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 00:02:04.301112 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79c2559-3270-48dc-90a9-3e00f4f658c3 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:04.301486 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdece14-b6cc-475b-b1c7-c22a8bdea6f4 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:04.349638 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acb87a4-ee58-4c32-98fe-163c02b72f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:04.350596 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acb87a4-ee58-4c32-98fe-163c02b72f8f None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:04.351067 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acb87a4-ee58-4c32-98fe-163c02b72f8f None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:04.351991 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acb87a4-ee58-4c32-98fe-163c02b72f8f None admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:04.352879 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acb87a4-ee58-4c32-98fe-163c02b72f8f None admin] RBAC: Adding query filter params (name=service) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:04.353370 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acb87a4-ee58-4c32-98fe-163c02b72f8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:04.354906 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acb87a4-ee58-4c32-98fe-163c02b72f8f None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:04.365007 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 190/373] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Oct 12 00:02:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:04.366589 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcffc0e9-0050-4a53-8ec2-5046ddd0ca52 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:04.367071 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcffc0e9-0050-4a53-8ec2-5046ddd0ca52 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:04.367672 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcffc0e9-0050-4a53-8ec2-5046ddd0ca52 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:04.369245 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 191/374] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:02:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:02:04.370241 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcffc0e9-0050-4a53-8ec2-5046ddd0ca52 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:04.446716 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdece14-b6cc-475b-b1c7-c22a8bdea6f4 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:04.450383 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:04.461501 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:04.481271 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e54e7f-07bf-4931-bcbf-fe2b4a6b02ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:04.481901 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e54e7f-07bf-4931-bcbf-fe2b4a6b02ec None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:04.482172 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e54e7f-07bf-4931-bcbf-fe2b4a6b02ec None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:04.482436 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e54e7f-07bf-4931-bcbf-fe2b4a6b02ec None admin] PATH_INFO: `/v3/users/nova` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:04.489349 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e54e7f-07bf-4931-bcbf-fe2b4a6b02ec None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:04.490774 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e54e7f-07bf-4931-bcbf-fe2b4a6b02ec None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:04.491956 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcffc0e9-0050-4a53-8ec2-5046ddd0ca52 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:04.503737 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-25e54e7f-07bf-4931-bcbf-fe2b4a6b02ec None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 12 00:02:04.503737 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:04.503737 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:04.503737 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:04.503737 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:04.503737 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:04.503737 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:04.503737 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:04.503737 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:04.503737 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:04.503737 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:04.503737 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:04.503737 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 00:02:04.503737 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 00:02:04.503737 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 00:02:04.503737 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 00:02:04.509568 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:04.509568 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:04.509568 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 00:02:04.509568 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 00:02:04.509568 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 00:02:04.509568 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 00:02:04.509568 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:04.509568 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:04.509568 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:04.509568 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:04.509568 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:04.509568 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:04.509568 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:04.509568 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:04.509568 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:04.509568 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:04.509568 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:04.509568 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:04.510722 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:04.510722 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:04.510722 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 00:02:04.510722 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 00:02:04.510722 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 00:02:04.510722 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 00:02:04.510722 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 00:02:04.510722 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 00:02:04.510722 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 12 00:02:04.510722 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:04.510722 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 184/375] 10.222.0.36 () {56 vars in 1093 bytes} [Wed Oct 12 00:02:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 00:02:04.510722 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee888da-dc95-4d0a-ae41-8e129940f6e1 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:04.510722 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee888da-dc95-4d0a-ae41-8e129940f6e1 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:04.511627 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee888da-dc95-4d0a-ae41-8e129940f6e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:04.525530 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4069a5-90a9-49f2-85ef-26a9be8336a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:04.526152 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4069a5-90a9-49f2-85ef-26a9be8336a0 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:04.526427 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4069a5-90a9-49f2-85ef-26a9be8336a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:04.526693 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4069a5-90a9-49f2-85ef-26a9be8336a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:04.527155 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-fee888da-dc95-4d0a-ae41-8e129940f6e1 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:02:04.528362 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-fee888da-dc95-4d0a-ae41-8e129940f6e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:02:04.538186 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4069a5-90a9-49f2-85ef-26a9be8336a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:04.539869 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4069a5-90a9-49f2-85ef-26a9be8336a0 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:04.555790 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-1b4069a5-90a9-49f2-85ef-26a9be8336a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:04.555790 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:04.555790 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:04.555790 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:04.555790 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:04.555790 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:04.555790 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:04.555790 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:04.555790 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:04.555790 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:04.555790 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:04.555790 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:04.555790 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 00:02:04.555790 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 00:02:04.555790 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 00:02:04.555790 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 00:02:04.556685 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:04.556685 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:04.556685 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:04.556685 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:04.556685 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:04.556685 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:04.556685 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:04.556685 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:04.556685 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:04.556685 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:04.556685 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:04.556685 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:04.556685 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:04.556685 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:04.556685 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:04.556685 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:04.556685 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 00:02:04.557571 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 00:02:04.557571 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:04.557571 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:04.557571 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 192/376] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:02:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 00:02:04.558024 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd71044-9b0d-4ad7-9672-6f72f5550b0d None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:04.558280 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd71044-9b0d-4ad7-9672-6f72f5550b0d None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:04.558548 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd71044-9b0d-4ad7-9672-6f72f5550b0d None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:04.559809 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 193/377] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:02:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:02:04.564296 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd71044-9b0d-4ad7-9672-6f72f5550b0d None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:04.599008 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 185/378] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:02:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 00:02:04.600235 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee888da-dc95-4d0a-ae41-8e129940f6e1 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:04.673405 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd71044-9b0d-4ad7-9672-6f72f5550b0d None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:04.687304 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:04.707584 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae8afb9-9da6-408a-b964-2f31dd986fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:04.708224 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae8afb9-9da6-408a-b964-2f31dd986fd0 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:04.708341 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee888da-dc95-4d0a-ae41-8e129940f6e1 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:04.708518 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae8afb9-9da6-408a-b964-2f31dd986fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:04.708742 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae8afb9-9da6-408a-b964-2f31dd986fd0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:04.711214 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:04.719150 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae8afb9-9da6-408a-b964-2f31dd986fd0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 00:02:04.724598 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-2ae8afb9-9da6-408a-b964-2f31dd986fd0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 12 00:02:04.724598 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:04.724598 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:04.724598 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:04.724598 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:04.724598 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:04.724598 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:04.724598 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:04.724598 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:04.724598 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:04.724598 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:04.724598 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:04.724598 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 00:02:04.724598 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 00:02:04.724598 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 00:02:04.724598 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 00:02:04.725567 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 00:02:04.725567 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 00:02:04.725567 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 00:02:04.725567 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 00:02:04.725567 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:04.725567 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:04.725567 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:04.725567 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:04.725567 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:04.725567 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:04.725567 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:04.725567 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:04.725567 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:04.725567 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:04.725567 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:04.725567 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:04.726767 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:04.726767 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:04.726767 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:04.726767 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:04.726767 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 00:02:04.726767 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 00:02:04.726767 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 00:02:04.726767 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 00:02:04.726767 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 00:02:04.726767 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 00:02:04.726767 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 12 00:02:04.726767 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:04.729346 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 194/379] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Oct 12 00:02:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:04.731744 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae8afb9-9da6-408a-b964-2f31dd986fd0 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:04.750171 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dc49f5-510d-4b75-aa72-20bcb5a25496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:04.750791 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc49f5-510d-4b75-aa72-20bcb5a25496 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:04.751286 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc49f5-510d-4b75-aa72-20bcb5a25496 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:04.751935 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc49f5-510d-4b75-aa72-20bcb5a25496 None admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:04.752656 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dc49f5-510d-4b75-aa72-20bcb5a25496 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:04.752957 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dc49f5-510d-4b75-aa72-20bcb5a25496 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:04.754910 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dc49f5-510d-4b75-aa72-20bcb5a25496 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:04.793037 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 186/380] 10.222.0.36 () {56 vars in 1133 bytes} [Wed Oct 12 00:02:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:04.794239 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c3d539-9dd8-4f19-812a-7b26995963ed None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:04.794522 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c3d539-9dd8-4f19-812a-7b26995963ed None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:04.794791 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c3d539-9dd8-4f19-812a-7b26995963ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:04.813204 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-f7c3d539-9dd8-4f19-812a-7b26995963ed None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:02:04.814658 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-f7c3d539-9dd8-4f19-812a-7b26995963ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:02:04.841581 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae8afb9-9da6-408a-b964-2f31dd986fd0 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:04.879269 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5429f6de-2b2a-4562-96bb-fd22889719de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:04.879835 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5429f6de-2b2a-4562-96bb-fd22889719de None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:04.880108 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5429f6de-2b2a-4562-96bb-fd22889719de None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:04.880378 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5429f6de-2b2a-4562-96bb-fd22889719de None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:04.881017 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5429f6de-2b2a-4562-96bb-fd22889719de None admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:04.881317 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5429f6de-2b2a-4562-96bb-fd22889719de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:04.882064 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 187/381] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:02:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 00:02:04.882693 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c3d539-9dd8-4f19-812a-7b26995963ed None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:04.882786 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5429f6de-2b2a-4562-96bb-fd22889719de None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:04.898083 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 195/382] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Oct 12 00:02:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:04.899243 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5429f6de-2b2a-4562-96bb-fd22889719de None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:04.933149 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:04.984679 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:04.993143 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c3d539-9dd8-4f19-812a-7b26995963ed None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:04.999649 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5429f6de-2b2a-4562-96bb-fd22889719de None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:05.028653 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8687cf-53a7-4117-8eb2-775a99256970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:05.029264 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8687cf-53a7-4117-8eb2-775a99256970 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:05.029556 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8687cf-53a7-4117-8eb2-775a99256970 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:05.029822 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8687cf-53a7-4117-8eb2-775a99256970 None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:05.030454 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8687cf-53a7-4117-8eb2-775a99256970 None admin] RBAC: Adding query filter params (name=service) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:05.030750 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8687cf-53a7-4117-8eb2-775a99256970 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:05.032132 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8687cf-53a7-4117-8eb2-775a99256970 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:05.036418 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8534a8bb-5658-48aa-bd83-60d1e16daecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:05.037043 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8534a8bb-5658-48aa-bd83-60d1e16daecc None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:05.037311 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8534a8bb-5658-48aa-bd83-60d1e16daecc None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:05.037575 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8534a8bb-5658-48aa-bd83-60d1e16daecc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:05.040263 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 188/383] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Oct 12 00:02:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:05.041029 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8687cf-53a7-4117-8eb2-775a99256970 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:05.052031 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8534a8bb-5658-48aa-bd83-60d1e16daecc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:05.053468 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8534a8bb-5658-48aa-bd83-60d1e16daecc None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:05.068776 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-8534a8bb-5658-48aa-bd83-60d1e16daecc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:05.068776 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:05.068776 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:05.068776 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:05.068776 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:05.068776 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:05.068776 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:05.068776 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:05.068776 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:05.068776 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:05.068776 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:05.068776 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:05.068776 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 00:02:05.068776 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 00:02:05.068776 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 00:02:05.068776 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 00:02:05.069693 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:05.069693 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:05.069693 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:05.069693 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:05.069693 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:05.069693 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:05.069693 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:05.069693 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:05.069693 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:05.069693 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:05.069693 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:05.069693 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:05.069693 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:05.069693 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:05.069693 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:05.069693 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:05.069693 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 00:02:05.070525 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 00:02:05.070525 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:05.070525 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:05.070525 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 196/384] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:02:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:05.071252 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8534a8bb-5658-48aa-bd83-60d1e16daecc None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:05.152175 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8687cf-53a7-4117-8eb2-775a99256970 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:05.157683 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:05.171939 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8534a8bb-5658-48aa-bd83-60d1e16daecc None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:05.184734 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c28598-0754-4331-9111-26aff41f6eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:05.185366 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c28598-0754-4331-9111-26aff41f6eda None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:05.185642 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c28598-0754-4331-9111-26aff41f6eda None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:05.185919 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c28598-0754-4331-9111-26aff41f6eda None admin] PATH_INFO: `/v3/roles/service` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:05.196383 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c28598-0754-4331-9111-26aff41f6eda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 00:02:05.201718 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-d2c28598-0754-4331-9111-26aff41f6eda None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 12 00:02:05.201718 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:05.201718 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:05.201718 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:05.201718 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:05.201718 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:05.201718 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:05.201718 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:05.201718 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:05.201718 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:05.201718 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:05.201718 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:05.201718 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 00:02:05.201718 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 00:02:05.201718 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 00:02:05.201718 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 00:02:05.202659 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 00:02:05.202659 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 00:02:05.202659 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 00:02:05.202659 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 00:02:05.202659 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:05.202659 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:05.202659 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:05.202659 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:05.202659 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:05.202659 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:05.202659 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:05.202659 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:05.202659 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:05.202659 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:05.202659 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:05.202659 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:05.203711 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:05.203711 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:05.203711 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:05.203711 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:05.203711 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 00:02:05.203711 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 00:02:05.203711 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 00:02:05.203711 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 00:02:05.203711 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 00:02:05.203711 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 00:02:05.203711 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 12 00:02:05.203711 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:05.203711 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 189/385] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Oct 12 00:02:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:05.203711 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c28598-0754-4331-9111-26aff41f6eda None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:05.208966 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec680d98-8f05-4bb3-9396-9e021ecef55e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:05.215623 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec680d98-8f05-4bb3-9396-9e021ecef55e None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:05.215623 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec680d98-8f05-4bb3-9396-9e021ecef55e None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:05.215623 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec680d98-8f05-4bb3-9396-9e021ecef55e None admin] PATH_INFO: `/v3/users/glance` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:05.217363 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec680d98-8f05-4bb3-9396-9e021ecef55e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:05.218960 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec680d98-8f05-4bb3-9396-9e021ecef55e None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:05.219572 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:05.231761 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-ec680d98-8f05-4bb3-9396-9e021ecef55e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 12 00:02:05.231761 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:05.231761 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:05.231761 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:05.231761 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:05.231761 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:05.231761 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:05.231761 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:05.231761 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:05.231761 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:05.231761 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:05.231761 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:05.231761 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 00:02:05.231761 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 00:02:05.231761 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 00:02:05.231761 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 00:02:05.238423 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:05.238423 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:05.238423 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 00:02:05.238423 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 00:02:05.238423 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 00:02:05.238423 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 00:02:05.238423 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:05.238423 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:05.238423 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:05.238423 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:05.238423 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:05.238423 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:05.238423 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:05.238423 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:05.238423 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:05.238423 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:05.238423 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:05.238423 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:05.239305 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:05.239305 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:05.239305 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 00:02:05.239305 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 00:02:05.239305 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 00:02:05.239305 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 00:02:05.239305 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 00:02:05.239305 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 00:02:05.239305 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 12 00:02:05.239305 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:05.239305 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 197/386] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Oct 12 00:02:05 2022] GET /identity/v3/users/glance => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:05.239305 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec680d98-8f05-4bb3-9396-9e021ecef55e None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:05.305644 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c28598-0754-4331-9111-26aff41f6eda None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:05.334371 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec680d98-8f05-4bb3-9396-9e021ecef55e None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:05.339557 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf81e85-5f4f-4baf-9e99-bfa7a71c46d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:05.340184 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf81e85-5f4f-4baf-9e99-bfa7a71c46d8 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:05.340456 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf81e85-5f4f-4baf-9e99-bfa7a71c46d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:05.340721 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf81e85-5f4f-4baf-9e99-bfa7a71c46d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:05.350222 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf81e85-5f4f-4baf-9e99-bfa7a71c46d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:05.351696 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf81e85-5f4f-4baf-9e99-bfa7a71c46d8 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:05.367446 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-daf81e85-5f4f-4baf-9e99-bfa7a71c46d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:05.367446 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:05.367446 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:05.367446 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:05.367446 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:05.367446 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:05.367446 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:05.367446 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:05.367446 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:05.367446 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:05.367446 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:05.367446 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:05.367446 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 00:02:05.367446 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 00:02:05.367446 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 00:02:05.367446 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 00:02:05.368453 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:05.368453 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:05.368453 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:05.368453 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:05.368453 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:05.368453 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:05.368453 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:05.368453 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:05.368453 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:05.368453 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:05.368453 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:05.368453 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:05.368453 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:05.368453 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:05.368453 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:05.368453 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:05.368453 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 00:02:05.369297 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 00:02:05.369297 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:05.369297 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:05.369297 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 190/387] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:02:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:05.369297 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf81e85-5f4f-4baf-9e99-bfa7a71c46d8 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:05.373813 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b860528f-1323-44bb-a0ee-7c2321ae7437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:05.374481 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b860528f-1323-44bb-a0ee-7c2321ae7437 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:05.374760 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b860528f-1323-44bb-a0ee-7c2321ae7437 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:05.375037 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b860528f-1323-44bb-a0ee-7c2321ae7437 None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:05.375756 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b860528f-1323-44bb-a0ee-7c2321ae7437 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:05.376063 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b860528f-1323-44bb-a0ee-7c2321ae7437 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:05.377466 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b860528f-1323-44bb-a0ee-7c2321ae7437 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:05.389933 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 198/388] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Oct 12 00:02:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:05.390713 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b860528f-1323-44bb-a0ee-7c2321ae7437 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:05.400279 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:05.437557 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:05.471747 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf81e85-5f4f-4baf-9e99-bfa7a71c46d8 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:05.491217 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b860528f-1323-44bb-a0ee-7c2321ae7437 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:05.505425 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449015a6-c017-49d4-a4aa-2c101ef43742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:05.506052 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449015a6-c017-49d4-a4aa-2c101ef43742 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:05.506327 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449015a6-c017-49d4-a4aa-2c101ef43742 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:05.506597 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449015a6-c017-49d4-a4aa-2c101ef43742 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:05.517238 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449015a6-c017-49d4-a4aa-2c101ef43742 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:05.518683 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449015a6-c017-49d4-a4aa-2c101ef43742 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:05.523635 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0872ec62-27dc-47c5-81f3-a68d68f8358a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:05.524240 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0872ec62-27dc-47c5-81f3-a68d68f8358a None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:05.524518 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0872ec62-27dc-47c5-81f3-a68d68f8358a None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:05.524793 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0872ec62-27dc-47c5-81f3-a68d68f8358a None admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:05.525448 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0872ec62-27dc-47c5-81f3-a68d68f8358a None admin] RBAC: Adding query filter params (name=service) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:05.525751 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0872ec62-27dc-47c5-81f3-a68d68f8358a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:05.527113 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0872ec62-27dc-47c5-81f3-a68d68f8358a None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:05.532545 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-449015a6-c017-49d4-a4aa-2c101ef43742 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:05.532545 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:05.532545 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:05.532545 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:05.532545 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:05.532545 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:05.532545 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:05.532545 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:05.532545 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:05.532545 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:05.532545 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:05.532545 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:05.532545 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 00:02:05.532545 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 00:02:05.532545 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 00:02:05.532545 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 00:02:05.533588 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:05.533588 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:05.533588 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:05.533588 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:05.533588 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:05.533588 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:05.533588 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:05.533588 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:05.533588 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:05.533588 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:05.533588 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:05.533588 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:05.533588 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:05.533588 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:05.533588 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:05.533588 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:05.533588 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 00:02:05.534456 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 00:02:05.534456 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:05.534456 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:05.534456 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 191/389] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:02:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:05.534456 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449015a6-c017-49d4-a4aa-2c101ef43742 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:05.541798 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 199/390] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Oct 12 00:02:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:05.541798 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0872ec62-27dc-47c5-81f3-a68d68f8358a None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:05.604021 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:05.636542 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449015a6-c017-49d4-a4aa-2c101ef43742 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:05.637679 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0872ec62-27dc-47c5-81f3-a68d68f8358a None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:05.652517 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:05.668924 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066f3fd3-af8e-4665-9860-37b2e48c94d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:05.669544 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066f3fd3-af8e-4665-9860-37b2e48c94d8 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:05.669813 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066f3fd3-af8e-4665-9860-37b2e48c94d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:05.670082 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066f3fd3-af8e-4665-9860-37b2e48c94d8 None admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:05.670797 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066f3fd3-af8e-4665-9860-37b2e48c94d8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:05.670894 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9482dff4-7c4f-4349-906c-0c5412e969db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:05.671096 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066f3fd3-af8e-4665-9860-37b2e48c94d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:05.671451 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9482dff4-7c4f-4349-906c-0c5412e969db None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:05.671743 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9482dff4-7c4f-4349-906c-0c5412e969db None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:05.672011 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9482dff4-7c4f-4349-906c-0c5412e969db None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:05.672554 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066f3fd3-af8e-4665-9860-37b2e48c94d8 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:05.672648 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9482dff4-7c4f-4349-906c-0c5412e969db None admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:05.672933 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9482dff4-7c4f-4349-906c-0c5412e969db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:05.674328 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9482dff4-7c4f-4349-906c-0c5412e969db None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:05.686770 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 200/391] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Oct 12 00:02:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:05.688120 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9482dff4-7c4f-4349-906c-0c5412e969db None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:05.690525 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 192/392] 10.222.0.36 () {56 vars in 1137 bytes} [Wed Oct 12 00:02:05 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:05.691380 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066f3fd3-af8e-4665-9860-37b2e48c94d8 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:05.790941 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066f3fd3-af8e-4665-9860-37b2e48c94d8 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:05.792924 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9482dff4-7c4f-4349-906c-0c5412e969db None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:05.819638 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:05.823915 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71843728-3bca-4c45-98f5-c72fee94159d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:05.824538 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71843728-3bca-4c45-98f5-c72fee94159d None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:05.824819 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71843728-3bca-4c45-98f5-c72fee94159d None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:05.825094 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71843728-3bca-4c45-98f5-c72fee94159d None admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:05.825733 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71843728-3bca-4c45-98f5-c72fee94159d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:05.826035 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71843728-3bca-4c45-98f5-c72fee94159d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:05.826991 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3079efc-2788-4efa-b2b3-92c58a2296d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:05.827437 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71843728-3bca-4c45-98f5-c72fee94159d None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:05.827645 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3079efc-2788-4efa-b2b3-92c58a2296d5 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:05.827924 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3079efc-2788-4efa-b2b3-92c58a2296d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:05.828193 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3079efc-2788-4efa-b2b3-92c58a2296d5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:05.835135 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3079efc-2788-4efa-b2b3-92c58a2296d5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:05.836591 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3079efc-2788-4efa-b2b3-92c58a2296d5 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:05.838907 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 193/393] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Oct 12 00:02:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:05.839747 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71843728-3bca-4c45-98f5-c72fee94159d None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:05.848640 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-d3079efc-2788-4efa-b2b3-92c58a2296d5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 00:02:05.848640 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:05.848640 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:05.848640 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:05.848640 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:05.848640 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:05.848640 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:05.848640 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:05.848640 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:05.848640 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:05.848640 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:05.848640 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:05.848640 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 00:02:05.848640 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 00:02:05.848640 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 00:02:05.849540 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 00:02:05.849540 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:05.849540 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:05.849540 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:05.849540 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:05.849540 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:05.849540 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:05.849540 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:05.849540 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:05.849540 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:05.849540 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:05.849540 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:05.849540 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:05.849540 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:05.849540 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:05.849540 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:05.849540 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:05.850369 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 00:02:05.850369 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 00:02:05.850369 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 00:02:05.850369 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 00:02:05.850369 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 00:02:05.850369 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 00:02:05.850369 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 00:02:05.850369 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:05.850369 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 201/394] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Oct 12 00:02:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:05.851308 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3079efc-2788-4efa-b2b3-92c58a2296d5 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:05.873491 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:05.936183 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71843728-3bca-4c45-98f5-c72fee94159d None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:05.952630 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3079efc-2788-4efa-b2b3-92c58a2296d5 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:05.968755 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03dc13c-4f83-4200-ba49-212e516504be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:05.969378 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03dc13c-4f83-4200-ba49-212e516504be None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:05.969655 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03dc13c-4f83-4200-ba49-212e516504be None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:05.969918 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03dc13c-4f83-4200-ba49-212e516504be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:05.980407 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03dc13c-4f83-4200-ba49-212e516504be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:05.981846 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03dc13c-4f83-4200-ba49-212e516504be None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:05.986494 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a0c219-091e-4e33-80f7-0d5521a53881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:05.987108 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a0c219-091e-4e33-80f7-0d5521a53881 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:05.987418 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a0c219-091e-4e33-80f7-0d5521a53881 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:05.987692 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a0c219-091e-4e33-80f7-0d5521a53881 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:05.995042 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a0c219-091e-4e33-80f7-0d5521a53881 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:05.996486 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a0c219-091e-4e33-80f7-0d5521a53881 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:05.996613 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-c03dc13c-4f83-4200-ba49-212e516504be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:05.996613 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:05.996613 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:05.996613 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:05.996613 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:05.996613 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:05.996613 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:05.996613 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:05.996613 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:05.996613 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:05.996613 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:05.996613 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:05.996613 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 00:02:05.996613 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 00:02:05.996613 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 00:02:05.996613 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 00:02:05.997606 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:05.997606 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:05.997606 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:05.997606 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:05.997606 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:05.997606 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:05.997606 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:05.997606 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:05.997606 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:05.997606 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:05.997606 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:05.997606 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:05.997606 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:05.997606 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:05.997606 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:05.997606 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:05.997606 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 00:02:05.998864 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 00:02:05.998864 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:05.998864 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:05.998864 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 194/395] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:02:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:05.998864 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03dc13c-4f83-4200-ba49-212e516504be None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:06.011503 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-13a0c219-091e-4e33-80f7-0d5521a53881 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 12 00:02:06.011503 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:06.011503 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:06.011503 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:06.011503 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:06.011503 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:06.011503 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:06.011503 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:06.011503 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:06.011503 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:06.011503 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:06.011503 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:06.011503 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 00:02:06.011503 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 00:02:06.011503 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 00:02:06.011503 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 00:02:06.012851 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:06.012851 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:06.012851 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 00:02:06.012851 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 00:02:06.012851 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 00:02:06.012851 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 00:02:06.012851 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:06.012851 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:06.012851 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:06.012851 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:06.012851 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:06.012851 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:06.012851 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:06.012851 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:06.012851 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:06.012851 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:06.012851 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:06.012851 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:06.013964 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:06.013964 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:06.013964 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 00:02:06.013964 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 00:02:06.013964 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 00:02:06.013964 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 00:02:06.013964 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 00:02:06.013964 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 00:02:06.013964 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 12 00:02:06.013964 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:06.017224 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 202/396] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Oct 12 00:02:05 2022] GET /identity/v3/users/cinder => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:06.017873 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a0c219-091e-4e33-80f7-0d5521a53881 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:06.065165 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:06.089684 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:06.102039 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03dc13c-4f83-4200-ba49-212e516504be None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:06.115008 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a0c219-091e-4e33-80f7-0d5521a53881 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:06.135264 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba64481-dadd-44c8-8737-eb165ac57428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:06.135930 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba64481-dadd-44c8-8737-eb165ac57428 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:06.136211 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba64481-dadd-44c8-8737-eb165ac57428 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:06.136485 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba64481-dadd-44c8-8737-eb165ac57428 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:06.146754 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba64481-dadd-44c8-8737-eb165ac57428 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:06.148029 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e5b149-93a2-4aa2-bfd6-66c86fb2b836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:06.148303 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba64481-dadd-44c8-8737-eb165ac57428 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:06.148622 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e5b149-93a2-4aa2-bfd6-66c86fb2b836 None admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:06.148896 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e5b149-93a2-4aa2-bfd6-66c86fb2b836 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:06.149159 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e5b149-93a2-4aa2-bfd6-66c86fb2b836 None admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:06.149788 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e5b149-93a2-4aa2-bfd6-66c86fb2b836 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:06.151205 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e5b149-93a2-4aa2-bfd6-66c86fb2b836 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:06.163159 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-aba64481-dadd-44c8-8737-eb165ac57428 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:06.163159 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:06.163159 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:06.163159 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:06.163159 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:06.163159 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:06.163159 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:06.163159 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:06.163159 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:06.163159 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:06.163159 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:06.163159 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:06.163159 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 00:02:06.163159 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 00:02:06.163159 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 00:02:06.163159 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 00:02:06.164334 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:06.164334 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:06.164334 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:06.164334 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:06.164334 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:06.164334 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:06.164334 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:06.164334 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:06.164334 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:06.164334 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:06.164334 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:06.164334 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:06.164334 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:06.164334 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:06.164334 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:06.164334 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:06.164334 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 00:02:06.165169 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 00:02:06.165169 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:06.165169 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:06.165169 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 195/397] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:02:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:06.165382 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba64481-dadd-44c8-8737-eb165ac57428 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:06.235235 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 203/398] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Oct 12 00:02:06 2022] POST /identity/v3/users => generated 247 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:02:06.235235 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e5b149-93a2-4aa2-bfd6-66c86fb2b836 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:06.333443 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba64481-dadd-44c8-8737-eb165ac57428 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:06.370732 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53370265-dee5-47eb-b24f-4197fed3c8bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:06.371353 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53370265-dee5-47eb-b24f-4197fed3c8bf None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:06.371656 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53370265-dee5-47eb-b24f-4197fed3c8bf None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:06.371927 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53370265-dee5-47eb-b24f-4197fed3c8bf None admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:06.372629 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53370265-dee5-47eb-b24f-4197fed3c8bf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:06.372935 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53370265-dee5-47eb-b24f-4197fed3c8bf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:06.374392 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53370265-dee5-47eb-b24f-4197fed3c8bf None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:06.375110 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e5b149-93a2-4aa2-bfd6-66c86fb2b836 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:06.379307 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:06.389941 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:06.398831 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 196/399] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Oct 12 00:02:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:06.399754 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53370265-dee5-47eb-b24f-4197fed3c8bf None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:06.414964 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589e3e53-6a3a-481d-bc57-753de0604cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:06.415610 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589e3e53-6a3a-481d-bc57-753de0604cd6 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:06.415886 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589e3e53-6a3a-481d-bc57-753de0604cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:06.419013 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589e3e53-6a3a-481d-bc57-753de0604cd6 None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:06.419013 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589e3e53-6a3a-481d-bc57-753de0604cd6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:06.419013 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589e3e53-6a3a-481d-bc57-753de0604cd6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:06.419013 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589e3e53-6a3a-481d-bc57-753de0604cd6 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:06.429713 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 204/400] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Oct 12 00:02:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:06.430999 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589e3e53-6a3a-481d-bc57-753de0604cd6 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:06.505057 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53370265-dee5-47eb-b24f-4197fed3c8bf None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:06.543356 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589e3e53-6a3a-481d-bc57-753de0604cd6 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:06.544971 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75afbb12-f695-403f-9c9e-61d9972a7aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:06.545587 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75afbb12-f695-403f-9c9e-61d9972a7aac None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:06.545860 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75afbb12-f695-403f-9c9e-61d9972a7aac None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:06.546125 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75afbb12-f695-403f-9c9e-61d9972a7aac None admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:06.546854 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75afbb12-f695-403f-9c9e-61d9972a7aac None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:06.547154 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75afbb12-f695-403f-9c9e-61d9972a7aac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:06.548650 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75afbb12-f695-403f-9c9e-61d9972a7aac None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:06.567417 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 197/401] 10.222.0.36 () {56 vars in 1137 bytes} [Wed Oct 12 00:02:06 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:06.568312 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75afbb12-f695-403f-9c9e-61d9972a7aac None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:06.587929 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a96fc6-c45d-480c-b184-ce046601727c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:06.588543 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a96fc6-c45d-480c-b184-ce046601727c None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:06.588814 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a96fc6-c45d-480c-b184-ce046601727c None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:06.589078 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a96fc6-c45d-480c-b184-ce046601727c None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:06.590134 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a96fc6-c45d-480c-b184-ce046601727c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:06.590436 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a96fc6-c45d-480c-b184-ce046601727c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:06.591834 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a96fc6-c45d-480c-b184-ce046601727c None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:06.609697 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 205/402] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Oct 12 00:02:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:06.609697 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a96fc6-c45d-480c-b184-ce046601727c None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:06.631047 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:06.636245 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:06.675390 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75afbb12-f695-403f-9c9e-61d9972a7aac None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:06.709544 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62d63d7-fcbb-4075-9e8a-4bad204be8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:06.710214 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62d63d7-fcbb-4075-9e8a-4bad204be8dd None admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:06.710493 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62d63d7-fcbb-4075-9e8a-4bad204be8dd None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:06.710762 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62d63d7-fcbb-4075-9e8a-4bad204be8dd None admin] PATH_INFO: `/v3/projects/0697927c6aac43d4b14a750fbbb1145e/users/a515b75aa3944912ad2364ffd7924f40/roles/a53c0c4b8b8645b6a1cfe96266daec8e` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:06.717134 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a96fc6-c45d-480c-b184-ce046601727c None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:06.728706 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62d63d7-fcbb-4075-9e8a-4bad204be8dd None admin] RBAC: Authorizing `identity:create_grant(project_id=0697927c6aac43d4b14a750fbbb1145e, user_id=a515b75aa3944912ad2364ffd7924f40, role_id=a53c0c4b8b8645b6a1cfe96266daec8e)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:06.730164 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62d63d7-fcbb-4075-9e8a-4bad204be8dd None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:06.751040 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97733dd-9afd-4830-8220-cde5287a5dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:06.755399 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97733dd-9afd-4830-8220-cde5287a5dd6 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:06.755399 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97733dd-9afd-4830-8220-cde5287a5dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:06.755399 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97733dd-9afd-4830-8220-cde5287a5dd6 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:06.759086 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97733dd-9afd-4830-8220-cde5287a5dd6 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:06.760775 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97733dd-9afd-4830-8220-cde5287a5dd6 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:06.773556 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-c97733dd-9afd-4830-8220-cde5287a5dd6 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 12 00:02:06.773556 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:06.773556 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:06.773556 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:06.773556 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:06.773556 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:06.773556 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:06.773556 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:06.773556 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:06.773556 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:06.773556 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:06.773556 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:06.773556 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 00:02:06.773556 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 00:02:06.773556 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 00:02:06.773556 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 00:02:06.774458 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:06.774458 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:06.774458 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 00:02:06.774458 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 00:02:06.774458 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 00:02:06.774458 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 00:02:06.774458 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:06.774458 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:06.774458 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:06.774458 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:06.774458 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:06.774458 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:06.774458 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:06.774458 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:06.774458 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:06.774458 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:06.774458 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:06.774458 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:06.775360 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:06.775360 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:06.775360 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 00:02:06.775360 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 00:02:06.775360 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 00:02:06.775360 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 00:02:06.775360 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 00:02:06.775360 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 00:02:06.775360 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 12 00:02:06.775360 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:06.779205 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 206/403] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Oct 12 00:02:06 2022] GET /identity/v3/users/neutron => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:06.779911 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97733dd-9afd-4830-8220-cde5287a5dd6 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:06.803853 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 198/404] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Oct 12 00:02:06 2022] PUT /identity/v3/projects/0697927c6aac43d4b14a750fbbb1145e/users/a515b75aa3944912ad2364ffd7924f40/roles/a53c0c4b8b8645b6a1cfe96266daec8e => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:02:06.804777 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62d63d7-fcbb-4075-9e8a-4bad204be8dd None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:06.894977 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:06.913801 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97733dd-9afd-4830-8220-cde5287a5dd6 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:06.946338 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62d63d7-fcbb-4075-9e8a-4bad204be8dd None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:06.958540 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9a8cc1-6423-4ccc-bcea-63d5b2cab27b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:06.958807 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:06.960101 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9a8cc1-6423-4ccc-bcea-63d5b2cab27b None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:06.960540 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9a8cc1-6423-4ccc-bcea-63d5b2cab27b None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:06.960885 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9a8cc1-6423-4ccc-bcea-63d5b2cab27b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:06.976336 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9a8cc1-6423-4ccc-bcea-63d5b2cab27b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:06.977980 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9a8cc1-6423-4ccc-bcea-63d5b2cab27b None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:06.995228 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31911c9a-e024-48b1-8b28-200290cc42cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:06.995910 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31911c9a-e024-48b1-8b28-200290cc42cf None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:06.996189 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31911c9a-e024-48b1-8b28-200290cc42cf None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:06.996469 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31911c9a-e024-48b1-8b28-200290cc42cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:06.998499 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-0d9a8cc1-6423-4ccc-bcea-63d5b2cab27b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:06.998499 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:06.998499 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:06.998499 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:06.998499 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:06.998499 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:06.998499 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:06.998499 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:06.998499 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:06.998499 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:06.998499 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:06.998499 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:06.998499 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 00:02:06.998499 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 00:02:06.998499 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 00:02:06.998499 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 00:02:06.999411 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:06.999411 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:06.999411 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:06.999411 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:06.999411 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:06.999411 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:06.999411 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:06.999411 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:06.999411 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:06.999411 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:06.999411 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:06.999411 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:06.999411 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:06.999411 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:06.999411 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:06.999411 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:06.999411 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 00:02:07.000255 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 00:02:07.000255 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:07.000255 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:07.000814 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 207/405] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:02:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:07.007192 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9a8cc1-6423-4ccc-bcea-63d5b2cab27b None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:07.011520 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31911c9a-e024-48b1-8b28-200290cc42cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:07.012994 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31911c9a-e024-48b1-8b28-200290cc42cf None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:07.029609 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-31911c9a-e024-48b1-8b28-200290cc42cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 00:02:07.029609 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:07.029609 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:07.029609 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:07.029609 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:07.029609 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:07.029609 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:07.029609 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:07.029609 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:07.029609 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:07.029609 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:07.029609 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:07.029609 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 00:02:07.029609 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 00:02:07.029609 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 00:02:07.030695 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 00:02:07.030695 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:07.030695 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:07.030695 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:07.030695 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:07.030695 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:07.030695 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:07.030695 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:07.030695 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:07.030695 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:07.030695 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:07.030695 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:07.030695 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:07.030695 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:07.030695 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:07.030695 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:07.030695 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:07.031794 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 00:02:07.031794 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 00:02:07.031794 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 00:02:07.031794 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 00:02:07.031794 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 00:02:07.031794 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 00:02:07.031794 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 00:02:07.031794 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:07.031794 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 199/406] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Oct 12 00:02:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:07.032371 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31911c9a-e024-48b1-8b28-200290cc42cf None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:07.129341 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9a8cc1-6423-4ccc-bcea-63d5b2cab27b None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:07.170008 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a944076d-a9cc-44a5-9715-e5d447ce9cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:07.170865 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a944076d-a9cc-44a5-9715-e5d447ce9cc2 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:07.171243 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a944076d-a9cc-44a5-9715-e5d447ce9cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:07.171637 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a944076d-a9cc-44a5-9715-e5d447ce9cc2 None admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:07.172246 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31911c9a-e024-48b1-8b28-200290cc42cf None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:07.172722 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a944076d-a9cc-44a5-9715-e5d447ce9cc2 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:07.172959 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a944076d-a9cc-44a5-9715-e5d447ce9cc2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:07.174519 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a944076d-a9cc-44a5-9715-e5d447ce9cc2 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:07.188045 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:07.214517 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1f475d-f7fe-4862-b7e3-d1c4e3ea1cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:07.215529 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1f475d-f7fe-4862-b7e3-d1c4e3ea1cc4 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:07.216009 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1f475d-f7fe-4862-b7e3-d1c4e3ea1cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:07.216476 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1f475d-f7fe-4862-b7e3-d1c4e3ea1cc4 None admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:07.216647 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 208/407] 10.222.0.36 () {56 vars in 1139 bytes} [Wed Oct 12 00:02:07 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:07.217516 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a944076d-a9cc-44a5-9715-e5d447ce9cc2 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:07.217820 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1f475d-f7fe-4862-b7e3-d1c4e3ea1cc4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:07.218403 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1f475d-f7fe-4862-b7e3-d1c4e3ea1cc4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:07.220253 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1f475d-f7fe-4862-b7e3-d1c4e3ea1cc4 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:07.242962 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 200/408] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Oct 12 00:02:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:07.242962 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1f475d-f7fe-4862-b7e3-d1c4e3ea1cc4 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:07.282482 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:07.340821 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a944076d-a9cc-44a5-9715-e5d447ce9cc2 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:07.382300 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1f475d-f7fe-4862-b7e3-d1c4e3ea1cc4 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:07.383206 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe015679-82ca-4bc0-b1de-1cc746fe1b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:07.383887 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe015679-82ca-4bc0-b1de-1cc746fe1b4d None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:07.384416 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe015679-82ca-4bc0-b1de-1cc746fe1b4d None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:07.384693 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe015679-82ca-4bc0-b1de-1cc746fe1b4d None admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:07.385394 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe015679-82ca-4bc0-b1de-1cc746fe1b4d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:07.385688 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe015679-82ca-4bc0-b1de-1cc746fe1b4d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:07.387300 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe015679-82ca-4bc0-b1de-1cc746fe1b4d None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:07.407478 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 209/409] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Oct 12 00:02:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:07.408551 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe015679-82ca-4bc0-b1de-1cc746fe1b4d None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:07.427202 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dfbbce-0ef9-4c08-997d-4a413c936aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:07.428441 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dfbbce-0ef9-4c08-997d-4a413c936aeb None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:07.428887 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dfbbce-0ef9-4c08-997d-4a413c936aeb None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:07.429514 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dfbbce-0ef9-4c08-997d-4a413c936aeb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:07.444098 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dfbbce-0ef9-4c08-997d-4a413c936aeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:07.445749 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dfbbce-0ef9-4c08-997d-4a413c936aeb None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:07.468311 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-85dfbbce-0ef9-4c08-997d-4a413c936aeb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:07.468311 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:07.468311 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:07.468311 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:07.468311 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:07.468311 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:07.468311 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:07.468311 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:07.468311 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:07.468311 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:07.468311 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:07.468311 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:07.468311 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 00:02:07.468311 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 00:02:07.468311 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 00:02:07.468311 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 00:02:07.469433 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:07.469433 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:07.469433 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:07.469433 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:07.469433 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:07.469433 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:07.469433 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:07.469433 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:07.469433 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:07.469433 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:07.469433 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:07.469433 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:07.469433 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:07.469433 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:07.469433 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:07.469433 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:07.469433 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 00:02:07.470494 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 00:02:07.470494 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:07.470494 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:07.473509 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 201/410] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:02:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:07.474489 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:07.474814 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dfbbce-0ef9-4c08-997d-4a413c936aeb None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:07.524439 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe015679-82ca-4bc0-b1de-1cc746fe1b4d None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:07.574040 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d127b3-bc96-43e0-99c8-81ff5e369559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:07.575104 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d127b3-bc96-43e0-99c8-81ff5e369559 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:07.575636 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d127b3-bc96-43e0-99c8-81ff5e369559 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:07.576118 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d127b3-bc96-43e0-99c8-81ff5e369559 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:07.577195 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:07.586190 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d127b3-bc96-43e0-99c8-81ff5e369559 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:07.588210 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d127b3-bc96-43e0-99c8-81ff5e369559 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:07.597943 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dfbbce-0ef9-4c08-997d-4a413c936aeb None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:07.602135 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-19d127b3-bc96-43e0-99c8-81ff5e369559 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 00:02:07.602135 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:07.602135 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:07.602135 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:07.602135 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:07.602135 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:07.602135 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:07.602135 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:07.602135 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:07.602135 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:07.602135 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:07.602135 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:07.602135 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 00:02:07.602135 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 00:02:07.602135 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 00:02:07.602981 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 00:02:07.602981 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:07.602981 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:07.602981 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:07.602981 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:07.602981 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:07.602981 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:07.602981 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:07.602981 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:07.602981 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:07.602981 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:07.602981 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:07.602981 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:07.602981 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:07.602981 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:07.602981 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:07.602981 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:07.603837 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 00:02:07.603837 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 00:02:07.603837 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 00:02:07.603837 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 00:02:07.603837 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 00:02:07.603837 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 00:02:07.603837 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 00:02:07.603837 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:07.610501 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 210/411] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Oct 12 00:02:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:07.611761 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d127b3-bc96-43e0-99c8-81ff5e369559 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:07.633757 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81c3522-d8b4-4a1b-8ab5-e629d97c406b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:07.634763 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81c3522-d8b4-4a1b-8ab5-e629d97c406b None admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:07.635390 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81c3522-d8b4-4a1b-8ab5-e629d97c406b None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:07.635390 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81c3522-d8b4-4a1b-8ab5-e629d97c406b None admin] PATH_INFO: `/v3/projects/0697927c6aac43d4b14a750fbbb1145e/users/59b883abbb674b3c976bacb64194a66d/roles/a53c0c4b8b8645b6a1cfe96266daec8e` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:07.651105 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81c3522-d8b4-4a1b-8ab5-e629d97c406b None admin] RBAC: Authorizing `identity:create_grant(project_id=0697927c6aac43d4b14a750fbbb1145e, user_id=59b883abbb674b3c976bacb64194a66d, role_id=a53c0c4b8b8645b6a1cfe96266daec8e)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:07.652673 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81c3522-d8b4-4a1b-8ab5-e629d97c406b None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:07.725983 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d127b3-bc96-43e0-99c8-81ff5e369559 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:07.726631 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 202/412] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Oct 12 00:02:07 2022] PUT /identity/v3/projects/0697927c6aac43d4b14a750fbbb1145e/users/59b883abbb674b3c976bacb64194a66d/roles/a53c0c4b8b8645b6a1cfe96266daec8e => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:02:07.727499 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81c3522-d8b4-4a1b-8ab5-e629d97c406b None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:07.747981 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:07.799814 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676cb93e-496b-4f86-8839-9f66f4f10961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:07.801340 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676cb93e-496b-4f86-8839-9f66f4f10961 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:07.802394 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676cb93e-496b-4f86-8839-9f66f4f10961 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:07.803091 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676cb93e-496b-4f86-8839-9f66f4f10961 None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:07.804647 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676cb93e-496b-4f86-8839-9f66f4f10961 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:07.806383 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676cb93e-496b-4f86-8839-9f66f4f10961 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:07.809588 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676cb93e-496b-4f86-8839-9f66f4f10961 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:07.836403 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 211/413] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Oct 12 00:02:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:07.841868 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676cb93e-496b-4f86-8839-9f66f4f10961 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:07.903256 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81c3522-d8b4-4a1b-8ab5-e629d97c406b None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:07.938949 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:07.957690 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ed0fe3-55d7-4905-9f6d-70efe49935e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:07.958269 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ed0fe3-55d7-4905-9f6d-70efe49935e9 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:07.958545 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ed0fe3-55d7-4905-9f6d-70efe49935e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:07.958814 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ed0fe3-55d7-4905-9f6d-70efe49935e9 None admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:07.959549 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ed0fe3-55d7-4905-9f6d-70efe49935e9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:07.959854 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ed0fe3-55d7-4905-9f6d-70efe49935e9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:07.961294 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ed0fe3-55d7-4905-9f6d-70efe49935e9 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:07.974301 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 203/414] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Oct 12 00:02:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:07.981077 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ed0fe3-55d7-4905-9f6d-70efe49935e9 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:08.014583 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676cb93e-496b-4f86-8839-9f66f4f10961 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:08.067473 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf71e87-4d1d-4a97-8ddd-2a62ec5d33c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:08.067473 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf71e87-4d1d-4a97-8ddd-2a62ec5d33c0 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:08.067473 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf71e87-4d1d-4a97-8ddd-2a62ec5d33c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:08.067473 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf71e87-4d1d-4a97-8ddd-2a62ec5d33c0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:08.092207 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf71e87-4d1d-4a97-8ddd-2a62ec5d33c0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:08.093678 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf71e87-4d1d-4a97-8ddd-2a62ec5d33c0 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:08.108134 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-7cf71e87-4d1d-4a97-8ddd-2a62ec5d33c0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 00:02:08.108134 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:08.108134 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:08.108134 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:08.108134 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:08.108134 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:08.108134 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:08.108134 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:08.108134 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:08.108134 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:08.108134 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:08.108134 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:08.108134 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 00:02:08.108134 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 00:02:08.108134 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 00:02:08.108995 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 00:02:08.108995 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:08.108995 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:08.108995 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:08.108995 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:08.108995 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:08.108995 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:08.108995 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:08.108995 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:08.108995 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:08.108995 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:08.108995 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:08.108995 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:08.108995 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:08.108995 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:08.108995 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:08.108995 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:08.109813 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 00:02:08.109813 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 00:02:08.109813 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 00:02:08.109813 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 00:02:08.109813 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 00:02:08.109813 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 00:02:08.109813 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 00:02:08.109813 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:08.110223 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ed0fe3-55d7-4905-9f6d-70efe49935e9 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:08.110301 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 212/415] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Oct 12 00:02:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:08.117736 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf71e87-4d1d-4a97-8ddd-2a62ec5d33c0 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:08.157284 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:08.172373 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26a3778-0c40-4080-b288-e7ff343a0e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:08.174268 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26a3778-0c40-4080-b288-e7ff343a0e7c None admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:08.179583 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26a3778-0c40-4080-b288-e7ff343a0e7c None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:08.179913 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26a3778-0c40-4080-b288-e7ff343a0e7c None admin] PATH_INFO: `/v3/projects/0697927c6aac43d4b14a750fbbb1145e/users/7f87febbbaa94626807a98da4a771ab1/roles/a53c0c4b8b8645b6a1cfe96266daec8e` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:08.202054 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26a3778-0c40-4080-b288-e7ff343a0e7c None admin] RBAC: Authorizing `identity:create_grant(project_id=0697927c6aac43d4b14a750fbbb1145e, user_id=7f87febbbaa94626807a98da4a771ab1, role_id=a53c0c4b8b8645b6a1cfe96266daec8e)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:08.203558 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26a3778-0c40-4080-b288-e7ff343a0e7c None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:08.226698 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf71e87-4d1d-4a97-8ddd-2a62ec5d33c0 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:08.262852 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dba7a87-21f0-4153-b586-68eaae53a328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:08.263487 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dba7a87-21f0-4153-b586-68eaae53a328 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:08.263764 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dba7a87-21f0-4153-b586-68eaae53a328 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:08.264027 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dba7a87-21f0-4153-b586-68eaae53a328 None admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:08.264708 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dba7a87-21f0-4153-b586-68eaae53a328 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:08.265005 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dba7a87-21f0-4153-b586-68eaae53a328 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:08.266463 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dba7a87-21f0-4153-b586-68eaae53a328 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:08.278948 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 213/416] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Oct 12 00:02:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:08.284370 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dba7a87-21f0-4153-b586-68eaae53a328 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:08.319529 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 204/417] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Oct 12 00:02:07 2022] PUT /identity/v3/projects/0697927c6aac43d4b14a750fbbb1145e/users/7f87febbbaa94626807a98da4a771ab1/roles/a53c0c4b8b8645b6a1cfe96266daec8e => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:02:08.417344 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:08.424725 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dba7a87-21f0-4153-b586-68eaae53a328 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:08.466931 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3b0627-fd17-44ed-8026-03940a71d669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:08.467645 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3b0627-fd17-44ed-8026-03940a71d669 None admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:08.467919 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3b0627-fd17-44ed-8026-03940a71d669 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:08.468187 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3b0627-fd17-44ed-8026-03940a71d669 None admin] PATH_INFO: `/v3/projects/0697927c6aac43d4b14a750fbbb1145e/users/51215dab618141b4991e232724094140/roles/a53c0c4b8b8645b6a1cfe96266daec8e` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:08.493462 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3b0627-fd17-44ed-8026-03940a71d669 None admin] RBAC: Authorizing `identity:create_grant(project_id=0697927c6aac43d4b14a750fbbb1145e, user_id=51215dab618141b4991e232724094140, role_id=a53c0c4b8b8645b6a1cfe96266daec8e)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:08.495522 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3b0627-fd17-44ed-8026-03940a71d669 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:08.500703 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02474a6-62df-4051-8908-cb0b6784a3cb None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:08.500989 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02474a6-62df-4051-8908-cb0b6784a3cb None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:08.501272 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02474a6-62df-4051-8908-cb0b6784a3cb None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:08.502735 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 205/418] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:02:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:02:08.507973 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4c6451-75b6-42ac-af0f-5660008ef02a None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:08.508455 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4c6451-75b6-42ac-af0f-5660008ef02a None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:08.512101 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4c6451-75b6-42ac-af0f-5660008ef02a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:08.531099 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-db4c6451-75b6-42ac-af0f-5660008ef02a None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:02:08.532569 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-db4c6451-75b6-42ac-af0f-5660008ef02a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:02:08.577249 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 214/419] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Oct 12 00:02:08 2022] PUT /identity/v3/projects/0697927c6aac43d4b14a750fbbb1145e/users/51215dab618141b4991e232724094140/roles/a53c0c4b8b8645b6a1cfe96266daec8e => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:02:08.584193 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:08.633013 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 206/420] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:02:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 12 00:02:08.934170 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3b0627-fd17-44ed-8026-03940a71d669 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:09.059137 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3b0627-fd17-44ed-8026-03940a71d669 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:09.105125 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31c69fb-fb58-4800-b20e-9da31282d477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:09.105764 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31c69fb-fb58-4800-b20e-9da31282d477 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:09.106093 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31c69fb-fb58-4800-b20e-9da31282d477 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:09.106367 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31c69fb-fb58-4800-b20e-9da31282d477 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:09.123495 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31c69fb-fb58-4800-b20e-9da31282d477 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 00:02:09.129010 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-c31c69fb-fb58-4800-b20e-9da31282d477 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 12 00:02:09.129010 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:09.129010 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:09.129010 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:09.129010 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:09.129010 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:09.129010 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:09.129010 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:09.129010 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:09.129010 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:09.129010 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:09.129010 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:09.129010 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 00:02:09.129010 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 00:02:09.129010 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 00:02:09.129010 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 00:02:09.129905 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 00:02:09.129905 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 00:02:09.129905 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 00:02:09.129905 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 00:02:09.129905 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:09.129905 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:09.129905 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:09.129905 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:09.129905 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:09.129905 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:09.129905 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:09.129905 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:09.129905 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:09.129905 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:09.129905 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:09.129905 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:09.130697 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:09.130697 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:09.130697 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:09.130697 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:09.130697 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 00:02:09.130697 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 00:02:09.130697 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 00:02:09.130697 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 00:02:09.130697 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 00:02:09.130697 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 00:02:09.130697 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 12 00:02:09.130697 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:09.130697 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 215/421] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Oct 12 00:02:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:09.137586 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4c6451-75b6-42ac-af0f-5660008ef02a None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:09.263395 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4c6451-75b6-42ac-af0f-5660008ef02a None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:09.312883 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbaeb42-931f-486f-a252-1b2f028f6235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:09.313531 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbaeb42-931f-486f-a252-1b2f028f6235 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:09.313813 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbaeb42-931f-486f-a252-1b2f028f6235 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:09.314088 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbaeb42-931f-486f-a252-1b2f028f6235 None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:09.314813 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbaeb42-931f-486f-a252-1b2f028f6235 None admin] RBAC: Adding query filter params (name=service) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:09.315115 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbaeb42-931f-486f-a252-1b2f028f6235 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:09.316764 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbaeb42-931f-486f-a252-1b2f028f6235 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:09.326660 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 207/422] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Oct 12 00:02:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:09.332300 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31c69fb-fb58-4800-b20e-9da31282d477 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:09.336738 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:09.421723 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c4127-19e9-4c60-bc7b-81fa47cd1204 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:09.422353 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c4127-19e9-4c60-bc7b-81fa47cd1204 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:09.422844 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c4127-19e9-4c60-bc7b-81fa47cd1204 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:09.424401 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 208/423] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:02:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 00:02:09.430611 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6039f68a-0d03-48e5-8816-96a1c870efdc None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:09.431152 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6039f68a-0d03-48e5-8816-96a1c870efdc None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:09.431656 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6039f68a-0d03-48e5-8816-96a1c870efdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:09.455582 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-6039f68a-0d03-48e5-8816-96a1c870efdc None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:02:09.458385 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-6039f68a-0d03-48e5-8816-96a1c870efdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:02:09.474807 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31c69fb-fb58-4800-b20e-9da31282d477 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:09.505340 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:09.541125 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effc83e2-855f-4c28-a6f4-12fb9134048b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:09.541125 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effc83e2-855f-4c28-a6f4-12fb9134048b None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:09.541125 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effc83e2-855f-4c28-a6f4-12fb9134048b None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:09.541125 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effc83e2-855f-4c28-a6f4-12fb9134048b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:09.565725 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 209/424] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:02:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 12 00:02:09.570824 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effc83e2-855f-4c28-a6f4-12fb9134048b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:09.570824 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effc83e2-855f-4c28-a6f4-12fb9134048b None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:09.595506 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-effc83e2-855f-4c28-a6f4-12fb9134048b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:09.595506 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:09.595506 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:09.595506 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:09.595506 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:09.595506 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:09.595506 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:09.595506 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:09.595506 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:09.595506 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:09.595506 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:09.595506 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:09.595506 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 00:02:09.595506 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 00:02:09.595506 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 00:02:09.595506 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 00:02:09.596474 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:09.596474 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:09.596474 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:09.596474 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:09.596474 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:09.596474 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:09.596474 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:09.596474 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:09.596474 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:09.596474 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:09.596474 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:09.596474 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:09.596474 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:09.596474 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:09.596474 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:09.596474 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:09.596474 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 00:02:09.597952 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 00:02:09.597952 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:09.597952 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:09.597952 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 216/425] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:02:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:09.602963 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6039f68a-0d03-48e5-8816-96a1c870efdc None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:09.723862 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6039f68a-0d03-48e5-8816-96a1c870efdc None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:09.758756 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea88b07-fbcb-4b01-97b7-eca9f0b8c9fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:09.759952 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea88b07-fbcb-4b01-97b7-eca9f0b8c9fb None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:09.760453 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea88b07-fbcb-4b01-97b7-eca9f0b8c9fb None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:09.760971 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea88b07-fbcb-4b01-97b7-eca9f0b8c9fb None admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:09.761809 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea88b07-fbcb-4b01-97b7-eca9f0b8c9fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:09.762297 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea88b07-fbcb-4b01-97b7-eca9f0b8c9fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:09.763874 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea88b07-fbcb-4b01-97b7-eca9f0b8c9fb None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:09.775913 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 210/426] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Oct 12 00:02:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:09.783415 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effc83e2-855f-4c28-a6f4-12fb9134048b None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:09.873703 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:09.893017 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea88b07-fbcb-4b01-97b7-eca9f0b8c9fb None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:09.944022 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:09.944225 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effc83e2-855f-4c28-a6f4-12fb9134048b None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:09.982847 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4655e288-19a6-4b1c-a24c-062ac136f52a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:09.983561 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4655e288-19a6-4b1c-a24c-062ac136f52a None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:09.983841 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4655e288-19a6-4b1c-a24c-062ac136f52a None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:09.984088 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4655e288-19a6-4b1c-a24c-062ac136f52a None admin] PATH_INFO: `/v3/users/swift` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:09.992932 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4655e288-19a6-4b1c-a24c-062ac136f52a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:09.994667 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4655e288-19a6-4b1c-a24c-062ac136f52a None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:10.007406 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-4655e288-19a6-4b1c-a24c-062ac136f52a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 12 00:02:10.007406 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:10.007406 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:10.007406 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:10.007406 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:10.007406 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:10.007406 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:10.007406 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:10.007406 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:10.007406 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:10.007406 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:10.007406 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:10.007406 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 00:02:10.007406 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 00:02:10.007406 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 00:02:10.007406 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 00:02:10.008492 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:10.008492 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:10.008492 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 00:02:10.008492 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 00:02:10.008492 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 00:02:10.008492 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 00:02:10.008492 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:10.008492 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:10.008492 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:10.008492 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:10.008492 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:10.008492 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:10.008492 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:10.008492 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:10.008492 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:10.008492 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:10.008492 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:10.008492 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:10.009969 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:10.009969 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:10.009969 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 00:02:10.009969 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 00:02:10.009969 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 00:02:10.009969 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 00:02:10.009969 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 00:02:10.009969 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 00:02:10.009969 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 12 00:02:10.009969 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:10.009969 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 217/427] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Oct 12 00:02:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:10.015157 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4655e288-19a6-4b1c-a24c-062ac136f52a None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:10.097737 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea88b07-fbcb-4b01-97b7-eca9f0b8c9fb None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:10.162748 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664d397c-7dc9-495b-bd4c-351b0481d2d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:10.163311 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664d397c-7dc9-495b-bd4c-351b0481d2d8 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:10.163628 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664d397c-7dc9-495b-bd4c-351b0481d2d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:10.163897 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664d397c-7dc9-495b-bd4c-351b0481d2d8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:10.181597 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4655e288-19a6-4b1c-a24c-062ac136f52a None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:10.184171 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664d397c-7dc9-495b-bd4c-351b0481d2d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 00:02:10.189133 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-664d397c-7dc9-495b-bd4c-351b0481d2d8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 12 00:02:10.189133 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:10.189133 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:10.189133 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:10.189133 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:10.189133 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:10.189133 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:10.189133 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:10.189133 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:10.189133 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:10.189133 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:10.189133 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:10.189133 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 00:02:10.189133 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 00:02:10.189133 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 00:02:10.189133 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 00:02:10.190013 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 00:02:10.190013 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 00:02:10.190013 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 00:02:10.190013 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 00:02:10.190013 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:10.190013 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:10.190013 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:10.190013 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:10.190013 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:10.190013 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:10.190013 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:10.190013 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:10.190013 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:10.190013 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:10.190013 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:10.190013 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:10.191935 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:10.191935 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:10.191935 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:10.191935 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:10.191935 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 00:02:10.191935 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 00:02:10.191935 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 00:02:10.191935 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 00:02:10.191935 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 00:02:10.191935 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 00:02:10.191935 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 12 00:02:10.191935 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:10.191935 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 211/428] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Oct 12 00:02:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 00:02:10.197389 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664d397c-7dc9-495b-bd4c-351b0481d2d8 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:10.209258 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:10.242592 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf24454f-9eed-46d3-9a5e-18a2428ec4db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:10.244675 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf24454f-9eed-46d3-9a5e-18a2428ec4db None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:10.244960 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf24454f-9eed-46d3-9a5e-18a2428ec4db None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:10.245230 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf24454f-9eed-46d3-9a5e-18a2428ec4db None admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:10.245963 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf24454f-9eed-46d3-9a5e-18a2428ec4db None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:10.246264 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf24454f-9eed-46d3-9a5e-18a2428ec4db None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:10.247739 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf24454f-9eed-46d3-9a5e-18a2428ec4db None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:10.290816 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 218/429] 10.222.0.36 () {56 vars in 1135 bytes} [Wed Oct 12 00:02:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:10.296400 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf24454f-9eed-46d3-9a5e-18a2428ec4db None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:10.349356 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:10.388156 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664d397c-7dc9-495b-bd4c-351b0481d2d8 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:10.456147 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39702f09-76d7-4075-8b2c-5092f23d9073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:10.456675 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39702f09-76d7-4075-8b2c-5092f23d9073 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:10.456950 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39702f09-76d7-4075-8b2c-5092f23d9073 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:10.457223 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39702f09-76d7-4075-8b2c-5092f23d9073 None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:10.457885 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39702f09-76d7-4075-8b2c-5092f23d9073 None admin] RBAC: Adding query filter params (name=service) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:10.458183 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39702f09-76d7-4075-8b2c-5092f23d9073 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:10.459585 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39702f09-76d7-4075-8b2c-5092f23d9073 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:10.475759 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 212/430] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Oct 12 00:02:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:10.477579 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d477a8-45c9-44b0-b8e9-784e72244d8d None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:10.477868 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d477a8-45c9-44b0-b8e9-784e72244d8d None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:10.478143 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d477a8-45c9-44b0-b8e9-784e72244d8d None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:10.483503 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 213/431] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:02:10 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:02:10.485302 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d477a8-45c9-44b0-b8e9-784e72244d8d None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:10.498774 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf24454f-9eed-46d3-9a5e-18a2428ec4db None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:10.541207 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b7fb19-6f17-47a8-bee5-09016f60e845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:10.541859 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b7fb19-6f17-47a8-bee5-09016f60e845 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:10.542134 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b7fb19-6f17-47a8-bee5-09016f60e845 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:10.542402 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b7fb19-6f17-47a8-bee5-09016f60e845 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:10.555765 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b7fb19-6f17-47a8-bee5-09016f60e845 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:10.557669 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b7fb19-6f17-47a8-bee5-09016f60e845 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:10.576877 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-94b7fb19-6f17-47a8-bee5-09016f60e845 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:10.576877 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:10.576877 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:10.576877 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:10.576877 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:10.576877 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:10.576877 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:10.576877 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:10.576877 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:10.576877 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:10.576877 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:10.576877 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:10.576877 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 00:02:10.576877 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 00:02:10.576877 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 00:02:10.576877 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 00:02:10.577813 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:10.577813 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:10.577813 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:10.577813 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:10.577813 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:10.577813 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:10.577813 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:10.577813 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:10.577813 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:10.577813 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:10.577813 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:10.577813 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:10.577813 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:10.577813 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:10.577813 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:10.577813 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:10.577813 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 00:02:10.578646 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 00:02:10.578646 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:10.578646 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:10.578646 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 219/432] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:02:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:10.583881 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec79466e-612b-4231-9263-7f2790d60d23 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:10.583881 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec79466e-612b-4231-9263-7f2790d60d23 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:10.583881 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec79466e-612b-4231-9263-7f2790d60d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:10.602047 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-ec79466e-612b-4231-9263-7f2790d60d23 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:02:10.603449 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-ec79466e-612b-4231-9263-7f2790d60d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:02:10.623733 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d477a8-45c9-44b0-b8e9-784e72244d8d None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:10.629251 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:10.665641 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c134229-47f5-4db2-a1c1-1d148b692d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:10.666743 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c134229-47f5-4db2-a1c1-1d148b692d61 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:10.667246 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c134229-47f5-4db2-a1c1-1d148b692d61 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:10.667757 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c134229-47f5-4db2-a1c1-1d148b692d61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:10.680385 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c134229-47f5-4db2-a1c1-1d148b692d61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:10.682418 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c134229-47f5-4db2-a1c1-1d148b692d61 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:10.691726 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:10.692359 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 220/433] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:02:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 00:02:10.699383 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec79466e-612b-4231-9263-7f2790d60d23 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:10.711454 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-2c134229-47f5-4db2-a1c1-1d148b692d61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:10.711454 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:10.711454 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:10.711454 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:10.711454 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:10.711454 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:10.711454 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:10.711454 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:10.711454 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:10.711454 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:10.711454 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:10.711454 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:10.711454 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 00:02:10.711454 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 00:02:10.711454 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 00:02:10.711454 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 00:02:10.712786 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:10.712786 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:10.712786 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:10.712786 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:10.712786 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:10.712786 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:10.712786 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:10.712786 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:10.712786 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:10.712786 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:10.712786 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:10.712786 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:10.712786 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:10.712786 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:10.712786 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:10.712786 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:10.712786 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 00:02:10.713691 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 00:02:10.713691 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:10.713691 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:10.714481 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 214/434] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:02:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:10.715667 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c134229-47f5-4db2-a1c1-1d148b692d61 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:10.800633 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec79466e-612b-4231-9263-7f2790d60d23 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:10.840592 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7120e9fb-1def-4279-9b6d-9aab637847bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:10.841385 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7120e9fb-1def-4279-9b6d-9aab637847bc None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:10.841779 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7120e9fb-1def-4279-9b6d-9aab637847bc None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:10.842148 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7120e9fb-1def-4279-9b6d-9aab637847bc None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:10.843052 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7120e9fb-1def-4279-9b6d-9aab637847bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:10.843493 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7120e9fb-1def-4279-9b6d-9aab637847bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:10.845099 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7120e9fb-1def-4279-9b6d-9aab637847bc None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:10.859393 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 221/435] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Oct 12 00:02:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:10.867523 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7120e9fb-1def-4279-9b6d-9aab637847bc None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:10.911729 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:10.912083 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c134229-47f5-4db2-a1c1-1d148b692d61 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:10.949395 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b23c3f-9186-42a8-9ebd-bad4599f14e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:10.950401 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b23c3f-9186-42a8-9ebd-bad4599f14e9 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:10.950905 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b23c3f-9186-42a8-9ebd-bad4599f14e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:10.951397 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b23c3f-9186-42a8-9ebd-bad4599f14e9 None admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:10.952299 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b23c3f-9186-42a8-9ebd-bad4599f14e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:10.952796 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b23c3f-9186-42a8-9ebd-bad4599f14e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:10.954483 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b23c3f-9186-42a8-9ebd-bad4599f14e9 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:10.967627 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 215/436] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Oct 12 00:02:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:10.969261 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b67eb7f-80c1-4a79-b316-97d5d7c065f0 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:10.969764 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b67eb7f-80c1-4a79-b316-97d5d7c065f0 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:10.970235 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b67eb7f-80c1-4a79-b316-97d5d7c065f0 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:10.972801 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 216/437] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:02:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:02:10.974102 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b67eb7f-80c1-4a79-b316-97d5d7c065f0 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:11.004892 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7120e9fb-1def-4279-9b6d-9aab637847bc None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:11.037648 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:11.044704 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d633e3d-09d6-4019-b8f3-3d047f6b9382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:11.045571 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d633e3d-09d6-4019-b8f3-3d047f6b9382 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:11.046120 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d633e3d-09d6-4019-b8f3-3d047f6b9382 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:11.046540 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d633e3d-09d6-4019-b8f3-3d047f6b9382 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:11.054915 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d633e3d-09d6-4019-b8f3-3d047f6b9382 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:11.056712 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d633e3d-09d6-4019-b8f3-3d047f6b9382 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:11.069149 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-5d633e3d-09d6-4019-b8f3-3d047f6b9382 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 00:02:11.069149 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:11.069149 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:11.069149 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:11.069149 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:11.069149 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:11.069149 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:11.069149 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:11.069149 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:11.069149 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:11.069149 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:11.069149 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:11.069149 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 00:02:11.069149 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 00:02:11.069149 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 00:02:11.070232 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 00:02:11.070232 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:11.070232 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:11.070232 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:11.070232 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:11.070232 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:11.070232 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:11.070232 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:11.070232 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:11.070232 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:11.070232 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:11.070232 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:11.070232 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:11.070232 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:11.070232 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:11.070232 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:11.070232 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:11.071218 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 00:02:11.071218 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 00:02:11.071218 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 00:02:11.071218 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 00:02:11.071218 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 00:02:11.071218 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 00:02:11.071218 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 00:02:11.071218 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:11.077968 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 222/438] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Oct 12 00:02:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:11.079769 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3107f4-3624-4a30-a87a-143f163537b8 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:11.080233 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3107f4-3624-4a30-a87a-143f163537b8 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:11.080651 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3107f4-3624-4a30-a87a-143f163537b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:11.092886 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b67eb7f-80c1-4a79-b316-97d5d7c065f0 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:11.102700 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-3c3107f4-3624-4a30-a87a-143f163537b8 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:02:11.104013 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-3c3107f4-3624-4a30-a87a-143f163537b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:02:11.131305 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7a9e35-bd5e-4b4e-a4a9-9c7ce63be1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:11.132074 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7a9e35-bd5e-4b4e-a4a9-9c7ce63be1ab None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:11.132415 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7a9e35-bd5e-4b4e-a4a9-9c7ce63be1ab None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:11.132750 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7a9e35-bd5e-4b4e-a4a9-9c7ce63be1ab None admin] PATH_INFO: `/v3/users/nova` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:11.140875 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7a9e35-bd5e-4b4e-a4a9-9c7ce63be1ab None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:11.142426 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7a9e35-bd5e-4b4e-a4a9-9c7ce63be1ab None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:11.153934 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-7b7a9e35-bd5e-4b4e-a4a9-9c7ce63be1ab None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 12 00:02:11.153934 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:11.153934 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:11.153934 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:11.153934 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:11.153934 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:11.153934 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:11.153934 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:11.153934 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:11.153934 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:11.153934 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:11.153934 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:11.153934 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 00:02:11.153934 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 00:02:11.153934 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 00:02:11.153934 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 00:02:11.155034 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:11.155034 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:11.155034 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 00:02:11.155034 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 00:02:11.155034 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 00:02:11.155034 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 00:02:11.155034 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:11.155034 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:11.155034 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:11.155034 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:11.155034 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:11.155034 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:11.155034 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:11.155034 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:11.155034 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:11.155034 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:11.155034 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:11.155034 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:11.156177 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:11.156177 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:11.156177 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 00:02:11.156177 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 00:02:11.156177 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 00:02:11.156177 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 00:02:11.156177 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 00:02:11.156177 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 00:02:11.156177 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 12 00:02:11.156177 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:11.161264 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 217/439] 10.222.0.36 () {56 vars in 1093 bytes} [Wed Oct 12 00:02:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:11.161264 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7a9e35-bd5e-4b4e-a4a9-9c7ce63be1ab None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:11.191414 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 223/440] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:02:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 00:02:11.191414 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3107f4-3624-4a30-a87a-143f163537b8 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:11.198940 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:11.292289 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3107f4-3624-4a30-a87a-143f163537b8 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:11.308050 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7a9e35-bd5e-4b4e-a4a9-9c7ce63be1ab None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:11.326071 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:11.328377 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb446e2-3e71-4529-8d79-8ba2af5df3ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:11.329009 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb446e2-3e71-4529-8d79-8ba2af5df3ea None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:11.329283 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb446e2-3e71-4529-8d79-8ba2af5df3ea None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:11.329551 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb446e2-3e71-4529-8d79-8ba2af5df3ea None admin] PATH_INFO: `/v3/roles/service` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:11.340087 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb446e2-3e71-4529-8d79-8ba2af5df3ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 00:02:11.344278 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9fde30-c181-4a73-bb5f-f4e6003b1e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:11.344878 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-cdb446e2-3e71-4529-8d79-8ba2af5df3ea None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 12 00:02:11.344878 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:11.344878 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:11.344878 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:11.344878 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:11.344878 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:11.344878 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:11.344878 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:11.344878 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:11.344878 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:11.344878 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:11.344878 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:11.344878 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 00:02:11.344878 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 00:02:11.344878 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 00:02:11.344878 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 00:02:11.345924 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 00:02:11.345924 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 00:02:11.345924 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 00:02:11.345924 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 00:02:11.345924 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:11.345924 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:11.345924 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:11.345924 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:11.345924 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:11.345924 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:11.345924 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:11.345924 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:11.345924 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:11.345924 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:11.345924 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:11.345924 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:11.347105 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:11.347105 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:11.347105 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:11.347105 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:11.347105 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 00:02:11.347105 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 00:02:11.347105 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 00:02:11.347105 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 00:02:11.347105 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 00:02:11.347105 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 00:02:11.347105 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 12 00:02:11.347105 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:11.347905 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9fde30-c181-4a73-bb5f-f4e6003b1e5c None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:11.347985 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 224/441] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Oct 12 00:02:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:11.347985 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb446e2-3e71-4529-8d79-8ba2af5df3ea None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:11.348123 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9fde30-c181-4a73-bb5f-f4e6003b1e5c None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:11.348123 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9fde30-c181-4a73-bb5f-f4e6003b1e5c None admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:11.348649 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9fde30-c181-4a73-bb5f-f4e6003b1e5c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:11.348974 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9fde30-c181-4a73-bb5f-f4e6003b1e5c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:11.350566 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9fde30-c181-4a73-bb5f-f4e6003b1e5c None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:11.366269 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 218/442] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Oct 12 00:02:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:11.367607 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e5c079-1175-44f6-892a-2c65a63fbe54 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:11.367984 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e5c079-1175-44f6-892a-2c65a63fbe54 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:11.368362 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e5c079-1175-44f6-892a-2c65a63fbe54 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:11.369808 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 219/443] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:02:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:02:11.374360 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e5c079-1175-44f6-892a-2c65a63fbe54 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:11.425671 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:11.463557 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb446e2-3e71-4529-8d79-8ba2af5df3ea None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:11.482013 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e5c079-1175-44f6-892a-2c65a63fbe54 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:11.498000 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cf75ac-6d4b-4701-9ffb-1c486555fb08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:11.498605 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cf75ac-6d4b-4701-9ffb-1c486555fb08 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:11.498884 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cf75ac-6d4b-4701-9ffb-1c486555fb08 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:11.499451 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cf75ac-6d4b-4701-9ffb-1c486555fb08 None admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:11.499897 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cf75ac-6d4b-4701-9ffb-1c486555fb08 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:11.500198 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cf75ac-6d4b-4701-9ffb-1c486555fb08 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:11.501628 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cf75ac-6d4b-4701-9ffb-1c486555fb08 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:11.516199 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5e8f0d-a841-4982-8ce0-de406565a693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:11.516794 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5e8f0d-a841-4982-8ce0-de406565a693 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:11.517069 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5e8f0d-a841-4982-8ce0-de406565a693 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:11.517335 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5e8f0d-a841-4982-8ce0-de406565a693 None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:11.517975 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5e8f0d-a841-4982-8ce0-de406565a693 None admin] RBAC: Adding query filter params (name=service) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:11.518272 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5e8f0d-a841-4982-8ce0-de406565a693 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:11.519210 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 225/444] 10.222.0.36 () {56 vars in 1133 bytes} [Wed Oct 12 00:02:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:11.519686 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5e8f0d-a841-4982-8ce0-de406565a693 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:11.520117 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cf75ac-6d4b-4701-9ffb-1c486555fb08 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:11.528026 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 220/445] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Oct 12 00:02:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:11.528880 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2102aca2-1dd4-4fce-a4ee-b64dea48b569 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:11.529157 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2102aca2-1dd4-4fce-a4ee-b64dea48b569 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:11.529424 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2102aca2-1dd4-4fce-a4ee-b64dea48b569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:11.547528 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-2102aca2-1dd4-4fce-a4ee-b64dea48b569 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:02:11.548662 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-2102aca2-1dd4-4fce-a4ee-b64dea48b569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:02:11.577336 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:11.614740 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 221/446] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:02:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 00:02:11.615714 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2102aca2-1dd4-4fce-a4ee-b64dea48b569 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:11.627574 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cf75ac-6d4b-4701-9ffb-1c486555fb08 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:11.661510 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed897e26-8c3c-4b5e-8dbe-eaf958dbd1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:11.662162 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed897e26-8c3c-4b5e-8dbe-eaf958dbd1ad None admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:11.662438 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed897e26-8c3c-4b5e-8dbe-eaf958dbd1ad None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:11.662706 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed897e26-8c3c-4b5e-8dbe-eaf958dbd1ad None admin] PATH_INFO: `/v3/projects/0697927c6aac43d4b14a750fbbb1145e/users/fc1bcffc78844dcbb1a9b7c80285249b/roles/a53c0c4b8b8645b6a1cfe96266daec8e` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:11.677332 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed897e26-8c3c-4b5e-8dbe-eaf958dbd1ad None admin] RBAC: Authorizing `identity:create_grant(project_id=0697927c6aac43d4b14a750fbbb1145e, user_id=fc1bcffc78844dcbb1a9b7c80285249b, role_id=a53c0c4b8b8645b6a1cfe96266daec8e)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:11.678905 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed897e26-8c3c-4b5e-8dbe-eaf958dbd1ad None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:11.681299 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:11.714500 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2102aca2-1dd4-4fce-a4ee-b64dea48b569 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:11.748992 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a825874-5cd2-4d8f-afb5-8405f171ffdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:11.749707 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a825874-5cd2-4d8f-afb5-8405f171ffdd None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:11.749988 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a825874-5cd2-4d8f-afb5-8405f171ffdd None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:11.750254 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a825874-5cd2-4d8f-afb5-8405f171ffdd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:11.763719 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 226/447] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Oct 12 00:02:11 2022] PUT /identity/v3/projects/0697927c6aac43d4b14a750fbbb1145e/users/fc1bcffc78844dcbb1a9b7c80285249b/roles/a53c0c4b8b8645b6a1cfe96266daec8e => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:02:11.763719 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed897e26-8c3c-4b5e-8dbe-eaf958dbd1ad None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:11.776029 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a825874-5cd2-4d8f-afb5-8405f171ffdd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 00:02:11.782158 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-0a825874-5cd2-4d8f-afb5-8405f171ffdd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 12 00:02:11.782158 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:11.782158 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:11.782158 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:11.782158 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:11.782158 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:11.782158 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:11.782158 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:11.782158 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:11.782158 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:11.782158 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:11.782158 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:11.782158 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 00:02:11.782158 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 00:02:11.782158 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 00:02:11.782158 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 00:02:11.784694 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 00:02:11.784694 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 00:02:11.784694 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 00:02:11.784694 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 00:02:11.784694 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:11.784694 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:11.784694 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:11.784694 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:11.784694 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:11.784694 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:11.784694 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:11.784694 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:11.784694 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:11.784694 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:11.784694 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:11.784694 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:11.786493 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:11.786493 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:11.786493 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:11.786493 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:11.786493 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 00:02:11.786493 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 00:02:11.786493 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 00:02:11.786493 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 00:02:11.786493 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 00:02:11.786493 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 00:02:11.786493 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 12 00:02:11.786493 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:11.786493 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 222/448] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Oct 12 00:02:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:11.788009 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a825874-5cd2-4d8f-afb5-8405f171ffdd None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:11.827735 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:11.893218 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed897e26-8c3c-4b5e-8dbe-eaf958dbd1ad None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:11.922539 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a825874-5cd2-4d8f-afb5-8405f171ffdd None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:11.930481 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6bf0c6-875f-4fc6-9164-d978c78006d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:11.931112 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6bf0c6-875f-4fc6-9164-d978c78006d6 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:11.931405 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6bf0c6-875f-4fc6-9164-d978c78006d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:11.931676 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6bf0c6-875f-4fc6-9164-d978c78006d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:11.961238 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a437246-a08c-4719-9d52-5f140c51317e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:11.961878 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a437246-a08c-4719-9d52-5f140c51317e None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:11.962157 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a437246-a08c-4719-9d52-5f140c51317e None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:11.962430 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a437246-a08c-4719-9d52-5f140c51317e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:11.974891 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a437246-a08c-4719-9d52-5f140c51317e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:11.976384 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a437246-a08c-4719-9d52-5f140c51317e None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:11.979877 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:11.991285 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-5a437246-a08c-4719-9d52-5f140c51317e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:11.991285 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:11.991285 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:11.991285 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:11.991285 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:11.991285 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:11.991285 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:11.991285 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:11.991285 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:11.991285 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:11.991285 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:11.991285 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:11.991285 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 00:02:11.991285 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 00:02:11.991285 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 00:02:11.991285 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 00:02:11.998626 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:11.998626 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:11.998626 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:11.998626 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:11.998626 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:11.998626 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:11.998626 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:11.998626 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:11.998626 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:11.998626 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:11.998626 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:11.998626 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:11.998626 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:11.998626 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:11.998626 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:11.998626 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:11.998626 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 00:02:11.999725 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 00:02:11.999725 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:11.999725 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:11.999725 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 223/449] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:02:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:11.999725 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a437246-a08c-4719-9d52-5f140c51317e None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:12.011186 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6bf0c6-875f-4fc6-9164-d978c78006d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:12.012698 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6bf0c6-875f-4fc6-9164-d978c78006d6 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:12.082885 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:12.103347 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a437246-a08c-4719-9d52-5f140c51317e None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:12.117903 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 227/450] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Oct 12 00:02:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 12 00:02:12.118775 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6bf0c6-875f-4fc6-9164-d978c78006d6 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:12.144281 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90e53d9-afce-45a4-b355-2cfe43943f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:12.144893 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e53d9-afce-45a4-b355-2cfe43943f99 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:12.145176 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e53d9-afce-45a4-b355-2cfe43943f99 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:12.145452 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e53d9-afce-45a4-b355-2cfe43943f99 None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:12.146115 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90e53d9-afce-45a4-b355-2cfe43943f99 None admin] RBAC: Adding query filter params (name=service) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:12.146415 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90e53d9-afce-45a4-b355-2cfe43943f99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:12.147819 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90e53d9-afce-45a4-b355-2cfe43943f99 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:12.155773 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 224/451] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Oct 12 00:02:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:12.156565 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90e53d9-afce-45a4-b355-2cfe43943f99 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:12.205365 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:12.237220 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6bf0c6-875f-4fc6-9164-d978c78006d6 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:12.268058 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90e53d9-afce-45a4-b355-2cfe43943f99 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:12.276321 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26edacab-f7d2-4b35-824f-9dc403fcc76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:12.276946 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26edacab-f7d2-4b35-824f-9dc403fcc76a None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:12.277224 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26edacab-f7d2-4b35-824f-9dc403fcc76a None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:12.277496 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26edacab-f7d2-4b35-824f-9dc403fcc76a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:12.287910 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26edacab-f7d2-4b35-824f-9dc403fcc76a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 00:02:12.293116 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-26edacab-f7d2-4b35-824f-9dc403fcc76a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 12 00:02:12.293116 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:12.293116 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:12.293116 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:12.293116 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:12.293116 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:12.293116 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:12.293116 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:12.293116 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:12.293116 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:12.293116 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:12.293116 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:12.293116 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 00:02:12.293116 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 00:02:12.293116 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 00:02:12.293116 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 00:02:12.294161 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 00:02:12.294161 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 00:02:12.294161 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 00:02:12.294161 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 00:02:12.294161 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:12.294161 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:12.294161 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:12.294161 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:12.294161 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:12.294161 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:12.294161 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:12.294161 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:12.294161 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:12.294161 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:12.294161 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:12.294161 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:12.295154 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:12.295154 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:12.295154 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:12.295154 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:12.295154 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 00:02:12.295154 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 00:02:12.295154 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 00:02:12.295154 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 00:02:12.295154 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 00:02:12.295154 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 00:02:12.295154 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 12 00:02:12.295154 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:12.295154 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 228/452] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Oct 12 00:02:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:12.295154 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26edacab-f7d2-4b35-824f-9dc403fcc76a None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:12.314283 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ab936f-3076-4d54-8b11-35b7df841978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:12.315122 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ab936f-3076-4d54-8b11-35b7df841978 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:12.315948 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ab936f-3076-4d54-8b11-35b7df841978 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:12.316226 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ab936f-3076-4d54-8b11-35b7df841978 None admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:12.316980 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ab936f-3076-4d54-8b11-35b7df841978 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:12.317282 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ab936f-3076-4d54-8b11-35b7df841978 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:12.318772 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ab936f-3076-4d54-8b11-35b7df841978 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:12.336243 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 225/453] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Oct 12 00:02:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:12.336243 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ab936f-3076-4d54-8b11-35b7df841978 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:12.336243 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:12.406770 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26edacab-f7d2-4b35-824f-9dc403fcc76a None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:12.438789 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ab936f-3076-4d54-8b11-35b7df841978 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:12.441276 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd91b136-5415-4832-a229-9c49f6ec5bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:12.441910 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd91b136-5415-4832-a229-9c49f6ec5bf8 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:12.442180 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd91b136-5415-4832-a229-9c49f6ec5bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:12.442448 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd91b136-5415-4832-a229-9c49f6ec5bf8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:12.444347 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:12.453029 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd91b136-5415-4832-a229-9c49f6ec5bf8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:12.454492 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd91b136-5415-4832-a229-9c49f6ec5bf8 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:12.469137 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-dd91b136-5415-4832-a229-9c49f6ec5bf8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:12.469137 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:12.469137 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:12.469137 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:12.469137 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:12.469137 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:12.469137 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:12.469137 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:12.469137 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:12.469137 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:12.469137 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:12.469137 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:12.469137 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 00:02:12.469137 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 00:02:12.469137 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 00:02:12.469137 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 00:02:12.470132 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:12.470132 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:12.470132 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:12.470132 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:12.470132 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:12.470132 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:12.470132 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:12.470132 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:12.470132 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:12.470132 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:12.470132 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:12.470132 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:12.470132 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:12.470132 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:12.470132 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:12.470132 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:12.470132 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 00:02:12.470957 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 00:02:12.470957 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:12.470957 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:12.470957 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 229/454] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:02:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:12.471165 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd91b136-5415-4832-a229-9c49f6ec5bf8 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:12.480840 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098a14bc-b7f4-42da-ab9d-be792068f2c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:12.481743 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098a14bc-b7f4-42da-ab9d-be792068f2c3 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:12.482194 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098a14bc-b7f4-42da-ab9d-be792068f2c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:12.482651 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098a14bc-b7f4-42da-ab9d-be792068f2c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:12.493505 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098a14bc-b7f4-42da-ab9d-be792068f2c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:12.495145 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098a14bc-b7f4-42da-ab9d-be792068f2c3 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:12.510391 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-098a14bc-b7f4-42da-ab9d-be792068f2c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:12.510391 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:12.510391 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:12.510391 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:12.510391 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:12.510391 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:12.510391 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:12.510391 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:12.510391 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:12.510391 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:12.510391 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:12.510391 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:12.510391 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 00:02:12.510391 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 00:02:12.510391 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 00:02:12.510391 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 00:02:12.511296 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:12.511296 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:12.511296 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:12.511296 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:12.511296 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:12.511296 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:12.511296 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:12.511296 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:12.511296 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:12.511296 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:12.511296 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:12.511296 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:12.511296 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:12.511296 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:12.511296 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:12.511296 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:12.511296 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 00:02:12.512305 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 00:02:12.512305 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:12.512305 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:12.513665 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 226/455] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:02:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:12.514569 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098a14bc-b7f4-42da-ab9d-be792068f2c3 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:12.582332 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:12.585721 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd91b136-5415-4832-a229-9c49f6ec5bf8 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:12.621086 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eda3644-d3b9-4c9b-82ed-d52547e65062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:12.621626 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda3644-d3b9-4c9b-82ed-d52547e65062 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:12.621898 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda3644-d3b9-4c9b-82ed-d52547e65062 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:12.622173 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda3644-d3b9-4c9b-82ed-d52547e65062 None admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:12.622824 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eda3644-d3b9-4c9b-82ed-d52547e65062 None admin] RBAC: Adding query filter params (name=service) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:12.623124 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eda3644-d3b9-4c9b-82ed-d52547e65062 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:12.624539 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eda3644-d3b9-4c9b-82ed-d52547e65062 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:12.630937 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098a14bc-b7f4-42da-ab9d-be792068f2c3 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:12.633296 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 230/456] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Oct 12 00:02:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:12.638682 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eda3644-d3b9-4c9b-82ed-d52547e65062 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:12.666697 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c600f8-04b6-409a-89cf-5a01bc7c7af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:12.667351 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c600f8-04b6-409a-89cf-5a01bc7c7af9 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:12.667662 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c600f8-04b6-409a-89cf-5a01bc7c7af9 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:12.667933 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c600f8-04b6-409a-89cf-5a01bc7c7af9 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:12.680226 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c600f8-04b6-409a-89cf-5a01bc7c7af9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:12.681717 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c600f8-04b6-409a-89cf-5a01bc7c7af9 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:12.689931 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:12.694142 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-58c600f8-04b6-409a-89cf-5a01bc7c7af9 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 12 00:02:12.694142 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:12.694142 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:12.694142 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:12.694142 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:12.694142 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:12.694142 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:12.694142 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:12.694142 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:12.694142 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:12.694142 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:12.694142 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:12.694142 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 00:02:12.694142 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 00:02:12.694142 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 00:02:12.694142 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 00:02:12.695005 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:12.695005 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:12.695005 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 00:02:12.695005 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 00:02:12.695005 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 00:02:12.695005 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 00:02:12.695005 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:12.695005 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:12.695005 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:12.695005 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:12.695005 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:12.695005 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:12.695005 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:12.695005 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:12.695005 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:12.695005 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:12.695005 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:12.695005 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:12.695931 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:12.695931 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:12.695931 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 00:02:12.695931 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 00:02:12.695931 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 00:02:12.695931 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 00:02:12.695931 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 00:02:12.695931 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 00:02:12.695931 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 12 00:02:12.695931 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:12.700083 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 227/457] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Oct 12 00:02:12 2022] GET /identity/v3/users/glance => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:12.700797 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c600f8-04b6-409a-89cf-5a01bc7c7af9 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:12.744255 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eda3644-d3b9-4c9b-82ed-d52547e65062 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:12.779643 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda80cea-ce12-4c87-a4b4-3dd9d1e30722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:12.780245 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda80cea-ce12-4c87-a4b4-3dd9d1e30722 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:12.780542 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda80cea-ce12-4c87-a4b4-3dd9d1e30722 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:12.780782 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda80cea-ce12-4c87-a4b4-3dd9d1e30722 None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:12.781435 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda80cea-ce12-4c87-a4b4-3dd9d1e30722 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:12.781708 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda80cea-ce12-4c87-a4b4-3dd9d1e30722 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:12.783073 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda80cea-ce12-4c87-a4b4-3dd9d1e30722 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:12.794753 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 231/458] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Oct 12 00:02:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:12.795554 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda80cea-ce12-4c87-a4b4-3dd9d1e30722 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:12.804882 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c600f8-04b6-409a-89cf-5a01bc7c7af9 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:12.818956 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:12.839759 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf800b5-d7aa-4459-a46f-2f2591959dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:12.840372 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf800b5-d7aa-4459-a46f-2f2591959dc8 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:12.840658 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf800b5-d7aa-4459-a46f-2f2591959dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:12.840925 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf800b5-d7aa-4459-a46f-2f2591959dc8 None admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:12.841964 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf800b5-d7aa-4459-a46f-2f2591959dc8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:12.842329 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf800b5-d7aa-4459-a46f-2f2591959dc8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:12.843985 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf800b5-d7aa-4459-a46f-2f2591959dc8 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:12.856027 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 228/459] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Oct 12 00:02:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:12.860708 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf800b5-d7aa-4459-a46f-2f2591959dc8 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:12.906646 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda80cea-ce12-4c87-a4b4-3dd9d1e30722 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:12.916217 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:12.943524 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3966e5-a574-4642-adaf-e10763f47a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:12.944572 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3966e5-a574-4642-adaf-e10763f47a6b None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:12.944856 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3966e5-a574-4642-adaf-e10763f47a6b None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:12.945402 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3966e5-a574-4642-adaf-e10763f47a6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:12.956773 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3966e5-a574-4642-adaf-e10763f47a6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:12.958031 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3966e5-a574-4642-adaf-e10763f47a6b None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:12.965342 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf800b5-d7aa-4459-a46f-2f2591959dc8 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:12.973950 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-ba3966e5-a574-4642-adaf-e10763f47a6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:12.973950 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:12.973950 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:12.973950 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:12.973950 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:12.973950 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:12.973950 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:12.973950 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:12.973950 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:12.973950 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:12.973950 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:12.973950 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:12.973950 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 00:02:12.973950 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 00:02:12.973950 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 00:02:12.973950 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 00:02:12.974835 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:12.974835 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:12.974835 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:12.974835 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:12.974835 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:12.974835 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:12.974835 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:12.974835 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:12.974835 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:12.974835 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:12.974835 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:12.974835 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:12.974835 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:12.974835 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:12.974835 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:12.974835 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:12.974835 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 00:02:12.975712 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 00:02:12.975712 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:12.975712 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:12.979692 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 232/460] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:02:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:12.980315 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3966e5-a574-4642-adaf-e10763f47a6b None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:13.004808 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160549dd-b12a-45ce-bae2-b6d983490d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:13.005502 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160549dd-b12a-45ce-bae2-b6d983490d1c None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:13.005846 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160549dd-b12a-45ce-bae2-b6d983490d1c None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:13.006181 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160549dd-b12a-45ce-bae2-b6d983490d1c None admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:13.006972 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160549dd-b12a-45ce-bae2-b6d983490d1c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:13.007337 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160549dd-b12a-45ce-bae2-b6d983490d1c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:13.008855 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160549dd-b12a-45ce-bae2-b6d983490d1c None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:13.026162 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 229/461] 10.222.0.36 () {56 vars in 1137 bytes} [Wed Oct 12 00:02:12 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:13.026972 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160549dd-b12a-45ce-bae2-b6d983490d1c None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:13.057521 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:13.090598 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3966e5-a574-4642-adaf-e10763f47a6b None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:13.125276 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160549dd-b12a-45ce-bae2-b6d983490d1c None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:13.126902 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ce1725-0893-4b25-8bd3-f153d659e631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:13.127580 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ce1725-0893-4b25-8bd3-f153d659e631 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:13.127888 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ce1725-0893-4b25-8bd3-f153d659e631 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:13.128239 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ce1725-0893-4b25-8bd3-f153d659e631 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:13.135055 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ce1725-0893-4b25-8bd3-f153d659e631 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:13.136575 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ce1725-0893-4b25-8bd3-f153d659e631 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:13.147711 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-c7ce1725-0893-4b25-8bd3-f153d659e631 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 00:02:13.147711 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:13.147711 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:13.147711 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:13.147711 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:13.147711 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:13.147711 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:13.147711 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:13.147711 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:13.147711 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:13.147711 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:13.147711 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:13.147711 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 00:02:13.147711 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 00:02:13.147711 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 00:02:13.148715 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 00:02:13.148715 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:13.148715 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:13.148715 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:13.148715 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:13.148715 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:13.148715 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:13.148715 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:13.148715 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:13.148715 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:13.148715 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:13.148715 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:13.148715 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:13.148715 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:13.148715 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:13.148715 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:13.148715 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:13.149916 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 00:02:13.149916 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 00:02:13.149916 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 00:02:13.149916 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 00:02:13.149916 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 00:02:13.149916 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 00:02:13.149916 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 00:02:13.149916 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:13.153382 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 233/462] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Oct 12 00:02:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:13.154154 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ce1725-0893-4b25-8bd3-f153d659e631 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:13.162577 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a61de8-fb34-411d-80da-c8440464f272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:13.163251 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a61de8-fb34-411d-80da-c8440464f272 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:13.163652 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a61de8-fb34-411d-80da-c8440464f272 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:13.164017 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a61de8-fb34-411d-80da-c8440464f272 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:13.171058 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a61de8-fb34-411d-80da-c8440464f272 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:13.172634 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a61de8-fb34-411d-80da-c8440464f272 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:13.177022 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:13.185136 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-66a61de8-fb34-411d-80da-c8440464f272 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 12 00:02:13.185136 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:13.185136 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:13.185136 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:13.185136 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:13.185136 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:13.185136 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:13.185136 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:13.185136 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:13.185136 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:13.185136 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:13.185136 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:13.185136 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 00:02:13.185136 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 00:02:13.185136 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 00:02:13.185136 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 00:02:13.186229 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:13.186229 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:13.186229 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 00:02:13.186229 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 00:02:13.186229 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 00:02:13.186229 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 00:02:13.186229 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:13.186229 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:13.186229 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:13.186229 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:13.186229 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:13.186229 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:13.186229 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:13.186229 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:13.186229 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:13.186229 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:13.186229 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:13.186229 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:13.187311 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:13.187311 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:13.187311 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 00:02:13.187311 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 00:02:13.187311 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 00:02:13.187311 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 00:02:13.187311 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 00:02:13.187311 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 00:02:13.187311 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 12 00:02:13.187311 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:13.188161 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 230/463] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Oct 12 00:02:13 2022] GET /identity/v3/users/cinder => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:13.188908 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a61de8-fb34-411d-80da-c8440464f272 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:13.263326 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ce1725-0893-4b25-8bd3-f153d659e631 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:13.289438 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:13.298536 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a61de8-fb34-411d-80da-c8440464f272 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:13.300627 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a69bb29-b285-4544-9abf-23aa582f9b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:13.301294 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a69bb29-b285-4544-9abf-23aa582f9b82 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:13.303495 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a69bb29-b285-4544-9abf-23aa582f9b82 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:13.303495 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a69bb29-b285-4544-9abf-23aa582f9b82 None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:13.303495 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a69bb29-b285-4544-9abf-23aa582f9b82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:13.303495 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a69bb29-b285-4544-9abf-23aa582f9b82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:13.304357 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a69bb29-b285-4544-9abf-23aa582f9b82 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:13.320525 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 234/464] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Oct 12 00:02:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:13.320525 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a69bb29-b285-4544-9abf-23aa582f9b82 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:13.338407 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e532c8b-b8bc-443f-a033-78280486a171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:13.339102 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e532c8b-b8bc-443f-a033-78280486a171 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:13.339486 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e532c8b-b8bc-443f-a033-78280486a171 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:13.339811 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e532c8b-b8bc-443f-a033-78280486a171 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:13.403443 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:13.406224 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e532c8b-b8bc-443f-a033-78280486a171 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:13.407706 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e532c8b-b8bc-443f-a033-78280486a171 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:13.424295 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a69bb29-b285-4544-9abf-23aa582f9b82 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:13.460153 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c31eaa-419b-4914-a6d5-fcf3bd32e204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:13.460753 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c31eaa-419b-4914-a6d5-fcf3bd32e204 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:13.461023 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c31eaa-419b-4914-a6d5-fcf3bd32e204 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:13.461289 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c31eaa-419b-4914-a6d5-fcf3bd32e204 None admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:13.461976 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c31eaa-419b-4914-a6d5-fcf3bd32e204 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:13.462262 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c31eaa-419b-4914-a6d5-fcf3bd32e204 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:13.463718 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c31eaa-419b-4914-a6d5-fcf3bd32e204 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:13.475460 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 235/465] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Oct 12 00:02:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:13.476319 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c31eaa-419b-4914-a6d5-fcf3bd32e204 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:13.511065 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:13.521518 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 231/466] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Oct 12 00:02:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 12 00:02:13.526264 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e532c8b-b8bc-443f-a033-78280486a171 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:13.592892 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c31eaa-419b-4914-a6d5-fcf3bd32e204 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:13.627065 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33a7138-379d-4c64-9896-38a6349eb850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:13.627697 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33a7138-379d-4c64-9896-38a6349eb850 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:13.628594 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33a7138-379d-4c64-9896-38a6349eb850 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:13.628865 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33a7138-379d-4c64-9896-38a6349eb850 None admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:13.629585 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33a7138-379d-4c64-9896-38a6349eb850 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:13.629884 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33a7138-379d-4c64-9896-38a6349eb850 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:13.631316 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33a7138-379d-4c64-9896-38a6349eb850 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:13.633256 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e532c8b-b8bc-443f-a033-78280486a171 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:13.640513 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:13.649171 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 236/467] 10.222.0.36 () {56 vars in 1137 bytes} [Wed Oct 12 00:02:13 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:13.654487 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33a7138-379d-4c64-9896-38a6349eb850 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:13.668591 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f70ff35-625b-4e51-b442-20a27070b799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:13.669997 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f70ff35-625b-4e51-b442-20a27070b799 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:13.670433 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f70ff35-625b-4e51-b442-20a27070b799 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:13.670738 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f70ff35-625b-4e51-b442-20a27070b799 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:13.678505 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f70ff35-625b-4e51-b442-20a27070b799 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:13.680142 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f70ff35-625b-4e51-b442-20a27070b799 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:13.692944 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-2f70ff35-625b-4e51-b442-20a27070b799 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 12 00:02:13.692944 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:13.692944 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:13.692944 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:13.692944 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:13.692944 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:13.692944 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:13.692944 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:13.692944 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:13.692944 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:13.692944 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:13.692944 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:13.692944 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 00:02:13.692944 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 00:02:13.692944 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 00:02:13.692944 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 00:02:13.694088 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:13.694088 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:13.694088 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 00:02:13.694088 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 00:02:13.694088 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 00:02:13.694088 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 00:02:13.694088 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:13.694088 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:13.694088 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:13.694088 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:13.694088 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:13.694088 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:13.694088 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:13.694088 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:13.694088 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:13.694088 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:13.694088 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:13.694088 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:13.695227 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:13.695227 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:13.695227 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 00:02:13.695227 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 00:02:13.695227 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 00:02:13.695227 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 00:02:13.695227 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 00:02:13.695227 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 00:02:13.695227 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 12 00:02:13.695227 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:13.700417 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 232/468] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Oct 12 00:02:13 2022] GET /identity/v3/users/neutron => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:13.700417 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f70ff35-625b-4e51-b442-20a27070b799 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:13.756423 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:13.761092 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33a7138-379d-4c64-9896-38a6349eb850 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:13.795734 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d4cbc9-9268-45d7-bd5a-705fd8f23b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:13.796586 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d4cbc9-9268-45d7-bd5a-705fd8f23b29 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:13.796823 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d4cbc9-9268-45d7-bd5a-705fd8f23b29 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:13.797150 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d4cbc9-9268-45d7-bd5a-705fd8f23b29 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:13.797939 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d4cbc9-9268-45d7-bd5a-705fd8f23b29 None admin] RBAC: Adding query filter params (role.id=a53c0c4b8b8645b6a1cfe96266daec8e, scope.project.id=0697927c6aac43d4b14a750fbbb1145e, user.id=a515b75aa3944912ad2364ffd7924f40) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:13.798306 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d4cbc9-9268-45d7-bd5a-705fd8f23b29 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:13.800396 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d4cbc9-9268-45d7-bd5a-705fd8f23b29 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:13.806554 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 237/469] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Oct 12 00:02:13 2022] GET /identity/v3/role_assignments?user.id=a515b75aa3944912ad2364ffd7924f40&scope.project.id=0697927c6aac43d4b14a750fbbb1145e&role.id=a53c0c4b8b8645b6a1cfe96266daec8e => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:13.807120 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f70ff35-625b-4e51-b442-20a27070b799 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:13.816417 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d4cbc9-9268-45d7-bd5a-705fd8f23b29 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:13.853323 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5e6054-b196-413e-8473-53c251f0d0a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:13.854876 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e6054-b196-413e-8473-53c251f0d0a8 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:13.855512 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e6054-b196-413e-8473-53c251f0d0a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:13.856092 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e6054-b196-413e-8473-53c251f0d0a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:13.873567 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5e6054-b196-413e-8473-53c251f0d0a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:13.876146 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5e6054-b196-413e-8473-53c251f0d0a8 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:13.899275 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-1c5e6054-b196-413e-8473-53c251f0d0a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:13.899275 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:13.899275 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:13.899275 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:13.899275 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:13.899275 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:13.899275 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:13.899275 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:13.899275 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:13.899275 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:13.899275 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:13.899275 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:13.899275 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 00:02:13.899275 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 00:02:13.899275 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 00:02:13.899275 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 00:02:13.900432 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:13.900432 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:13.900432 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:13.900432 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:13.900432 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:13.900432 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:13.900432 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:13.900432 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:13.900432 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:13.900432 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:13.900432 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:13.900432 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:13.900432 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:13.900432 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:13.900432 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:13.900432 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:13.900432 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 00:02:13.901449 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 00:02:13.901449 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:13.901449 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:13.902383 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:13.904753 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 233/470] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:02:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:13.905931 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5e6054-b196-413e-8473-53c251f0d0a8 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:13.961601 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d4cbc9-9268-45d7-bd5a-705fd8f23b29 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:14.006351 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9465fc-d762-46d0-9a86-6837fbdda701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:14.006960 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9465fc-d762-46d0-9a86-6837fbdda701 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:14.007621 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9465fc-d762-46d0-9a86-6837fbdda701 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:14.007921 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9465fc-d762-46d0-9a86-6837fbdda701 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:14.021095 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5e6054-b196-413e-8473-53c251f0d0a8 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:14.024369 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:14.056834 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1928aac5-2c8c-4e99-8c1b-4bef3c342698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:14.057476 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1928aac5-2c8c-4e99-8c1b-4bef3c342698 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:14.057761 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1928aac5-2c8c-4e99-8c1b-4bef3c342698 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:14.058035 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1928aac5-2c8c-4e99-8c1b-4bef3c342698 None admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:14.058766 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1928aac5-2c8c-4e99-8c1b-4bef3c342698 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:14.059076 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1928aac5-2c8c-4e99-8c1b-4bef3c342698 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:14.060654 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1928aac5-2c8c-4e99-8c1b-4bef3c342698 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:14.081507 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 234/471] 10.222.0.36 () {56 vars in 1139 bytes} [Wed Oct 12 00:02:13 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:14.082177 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1928aac5-2c8c-4e99-8c1b-4bef3c342698 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:14.086498 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9465fc-d762-46d0-9a86-6837fbdda701 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:14.088031 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9465fc-d762-46d0-9a86-6837fbdda701 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:14.152071 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:14.185455 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1928aac5-2c8c-4e99-8c1b-4bef3c342698 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:14.196780 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 238/472] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Oct 12 00:02:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 12 00:02:14.196780 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa024cb3-403e-45cb-af5b-200d4d63fa58 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:14.196780 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa024cb3-403e-45cb-af5b-200d4d63fa58 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:14.196780 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa024cb3-403e-45cb-af5b-200d4d63fa58 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:14.200371 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 239/473] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:02:14 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:02:14.201375 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa024cb3-403e-45cb-af5b-200d4d63fa58 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:14.226272 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0f4f69-0965-4322-93cc-1e4dac59cab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:14.226808 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f4f69-0965-4322-93cc-1e4dac59cab9 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:14.227086 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f4f69-0965-4322-93cc-1e4dac59cab9 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:14.227357 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f4f69-0965-4322-93cc-1e4dac59cab9 None admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:14.228026 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0f4f69-0965-4322-93cc-1e4dac59cab9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:14.228329 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0f4f69-0965-4322-93cc-1e4dac59cab9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:14.229705 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0f4f69-0965-4322-93cc-1e4dac59cab9 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:14.242017 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 235/474] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Oct 12 00:02:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:14.243131 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0f4f69-0965-4322-93cc-1e4dac59cab9 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:14.274685 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:14.311717 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa024cb3-403e-45cb-af5b-200d4d63fa58 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:14.347137 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feb4a59-10e5-4eca-a3cb-18fdc91aa826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:14.347813 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feb4a59-10e5-4eca-a3cb-18fdc91aa826 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:14.348093 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feb4a59-10e5-4eca-a3cb-18fdc91aa826 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:14.348367 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feb4a59-10e5-4eca-a3cb-18fdc91aa826 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:14.358317 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0f4f69-0965-4322-93cc-1e4dac59cab9 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:14.380767 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:14.395020 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb930814-b724-468a-9f8e-44d2085e9aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:14.395677 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb930814-b724-468a-9f8e-44d2085e9aeb None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:14.395949 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb930814-b724-468a-9f8e-44d2085e9aeb None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:14.396217 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb930814-b724-468a-9f8e-44d2085e9aeb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:14.406754 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb930814-b724-468a-9f8e-44d2085e9aeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:14.408968 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb930814-b724-468a-9f8e-44d2085e9aeb None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:14.422475 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feb4a59-10e5-4eca-a3cb-18fdc91aa826 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:14.423971 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feb4a59-10e5-4eca-a3cb-18fdc91aa826 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:14.428198 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-eb930814-b724-468a-9f8e-44d2085e9aeb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:14.428198 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:14.428198 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:14.428198 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:14.428198 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:14.428198 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:14.428198 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:14.428198 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:14.428198 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:14.428198 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:14.428198 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:14.428198 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:14.428198 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 00:02:14.428198 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 00:02:14.428198 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 00:02:14.428198 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 00:02:14.429238 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:14.429238 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:14.429238 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:14.429238 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:14.429238 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:14.429238 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:14.429238 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:14.429238 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:14.429238 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:14.429238 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:14.429238 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:14.429238 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:14.429238 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:14.429238 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:14.429238 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:14.429238 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:14.429238 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 00:02:14.430300 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 00:02:14.430300 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:14.430300 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:14.432064 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 236/475] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:02:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:14.433711 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9db06fe-7387-4596-8b06-181b12a5b142 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:14.434142 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9db06fe-7387-4596-8b06-181b12a5b142 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:14.434841 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9db06fe-7387-4596-8b06-181b12a5b142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:14.454494 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-d9db06fe-7387-4596-8b06-181b12a5b142 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:02:14.455741 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-d9db06fe-7387-4596-8b06-181b12a5b142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:02:14.531768 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 237/476] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:02:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 00:02:14.531768 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9db06fe-7387-4596-8b06-181b12a5b142 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:14.532760 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:14.548177 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 240/477] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Oct 12 00:02:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 12 00:02:14.549283 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feb4a59-10e5-4eca-a3cb-18fdc91aa826 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:14.615642 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:14.658747 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feb4a59-10e5-4eca-a3cb-18fdc91aa826 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:14.659127 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9db06fe-7387-4596-8b06-181b12a5b142 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:14.695971 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05cb7ad-8715-4198-b4f8-7c572a469c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:14.696702 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05cb7ad-8715-4198-b4f8-7c572a469c5b None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:14.696977 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05cb7ad-8715-4198-b4f8-7c572a469c5b None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:14.697245 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05cb7ad-8715-4198-b4f8-7c572a469c5b None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:14.697880 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05cb7ad-8715-4198-b4f8-7c572a469c5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:14.698181 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05cb7ad-8715-4198-b4f8-7c572a469c5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:14.699598 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05cb7ad-8715-4198-b4f8-7c572a469c5b None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:14.700957 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b242eb6-4ecf-4604-9cef-610bb9109292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:14.702788 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b242eb6-4ecf-4604-9cef-610bb9109292 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:14.703330 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b242eb6-4ecf-4604-9cef-610bb9109292 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:14.703826 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b242eb6-4ecf-4604-9cef-610bb9109292 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:14.714256 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b242eb6-4ecf-4604-9cef-610bb9109292 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:14.716233 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b242eb6-4ecf-4604-9cef-610bb9109292 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:14.719097 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 241/478] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Oct 12 00:02:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:14.720172 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05cb7ad-8715-4198-b4f8-7c572a469c5b None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:14.732151 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-5b242eb6-4ecf-4604-9cef-610bb9109292 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 00:02:14.732151 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:14.732151 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:14.732151 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:14.732151 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:14.732151 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:14.732151 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:14.732151 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:14.732151 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:14.732151 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:14.732151 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:14.732151 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:14.732151 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 00:02:14.732151 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 00:02:14.732151 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 00:02:14.732989 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 00:02:14.732989 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:14.732989 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:14.732989 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:14.732989 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:14.732989 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:14.732989 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:14.732989 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:14.732989 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:14.732989 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:14.732989 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:14.732989 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:14.732989 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:14.732989 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:14.732989 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:14.732989 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:14.732989 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:14.733938 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 00:02:14.733938 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 00:02:14.733938 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 00:02:14.733938 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 00:02:14.733938 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 00:02:14.733938 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 00:02:14.733938 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 00:02:14.733938 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:14.736967 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 238/479] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Oct 12 00:02:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:14.738618 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b242eb6-4ecf-4604-9cef-610bb9109292 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:14.818831 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:14.837501 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05cb7ad-8715-4198-b4f8-7c572a469c5b None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:14.862141 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:14.866423 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b242eb6-4ecf-4604-9cef-610bb9109292 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:14.874114 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb870d6-af0f-48ce-ba73-d70758018d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:14.874742 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb870d6-af0f-48ce-ba73-d70758018d42 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:14.875014 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb870d6-af0f-48ce-ba73-d70758018d42 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:14.875650 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb870d6-af0f-48ce-ba73-d70758018d42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:14.887254 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb870d6-af0f-48ce-ba73-d70758018d42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:14.889258 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb870d6-af0f-48ce-ba73-d70758018d42 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:14.901196 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eb49aa-e5d9-4a21-a77f-db374dad36b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:14.901822 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eb49aa-e5d9-4a21-a77f-db374dad36b7 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:14.902150 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eb49aa-e5d9-4a21-a77f-db374dad36b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:14.902416 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eb49aa-e5d9-4a21-a77f-db374dad36b7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:14.906479 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-7bb870d6-af0f-48ce-ba73-d70758018d42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:14.906479 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:14.906479 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:14.906479 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:14.906479 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:14.906479 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:14.906479 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:14.906479 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:14.906479 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:14.906479 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:14.906479 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:14.906479 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:14.906479 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 00:02:14.906479 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 00:02:14.906479 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 00:02:14.906479 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 00:02:14.907338 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:14.907338 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:14.907338 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:14.907338 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:14.907338 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:14.907338 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:14.907338 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:14.907338 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:14.907338 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:14.907338 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:14.907338 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:14.907338 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:14.907338 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:14.907338 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:14.907338 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:14.907338 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:14.907338 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 00:02:14.908215 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 00:02:14.908215 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:14.908215 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:14.908215 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 242/480] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:02:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:14.910473 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eb49aa-e5d9-4a21-a77f-db374dad36b7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:14.911965 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eb49aa-e5d9-4a21-a77f-db374dad36b7 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:14.912463 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb870d6-af0f-48ce-ba73-d70758018d42 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:14.924667 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-98eb49aa-e5d9-4a21-a77f-db374dad36b7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 00:02:14.924667 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:14.924667 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:14.924667 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:14.924667 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:14.924667 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:14.924667 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:14.924667 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:14.924667 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:14.924667 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:14.924667 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:14.924667 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:14.924667 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 00:02:14.924667 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 00:02:14.924667 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 00:02:14.925564 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 00:02:14.925564 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:14.925564 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:14.925564 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:14.925564 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:14.925564 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:14.925564 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:14.925564 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:14.925564 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:14.925564 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:14.925564 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:14.925564 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:14.925564 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:14.925564 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:14.925564 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:14.925564 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:14.925564 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:14.926992 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 00:02:14.926992 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 00:02:14.926992 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 00:02:14.926992 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 00:02:14.926992 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 00:02:14.926992 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 00:02:14.926992 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 00:02:14.926992 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:14.926992 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 239/481] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Oct 12 00:02:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:14.926992 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eb49aa-e5d9-4a21-a77f-db374dad36b7 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:15.024660 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb870d6-af0f-48ce-ba73-d70758018d42 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:15.029377 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eb49aa-e5d9-4a21-a77f-db374dad36b7 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:15.055419 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:15.061313 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5ec9a3-12a6-4fed-8094-baea456d7d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:15.062144 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5ec9a3-12a6-4fed-8094-baea456d7d85 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:15.067229 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5ec9a3-12a6-4fed-8094-baea456d7d85 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:15.067229 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5ec9a3-12a6-4fed-8094-baea456d7d85 None admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:15.067229 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5ec9a3-12a6-4fed-8094-baea456d7d85 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:15.067229 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5ec9a3-12a6-4fed-8094-baea456d7d85 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:15.067229 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5ec9a3-12a6-4fed-8094-baea456d7d85 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:15.067704 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fff764f-94e6-4eab-93db-787a74755462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:15.067704 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fff764f-94e6-4eab-93db-787a74755462 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:15.067704 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fff764f-94e6-4eab-93db-787a74755462 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:15.067901 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fff764f-94e6-4eab-93db-787a74755462 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:15.078486 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fff764f-94e6-4eab-93db-787a74755462 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 00:02:15.082712 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 243/482] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Oct 12 00:02:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:15.083428 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-4fff764f-94e6-4eab-93db-787a74755462 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 12 00:02:15.083428 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:15.083428 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:15.083428 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:15.083428 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:15.083428 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:15.083428 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:15.083428 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:15.083428 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:15.083428 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:15.083428 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:15.083428 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:15.083428 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 00:02:15.083428 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 00:02:15.083428 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 00:02:15.083428 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 00:02:15.084316 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 00:02:15.084316 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 00:02:15.084316 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 00:02:15.084316 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 00:02:15.084316 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:15.084316 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:15.084316 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:15.084316 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:15.084316 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:15.084316 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:15.084316 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:15.084316 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:15.084316 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:15.084316 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:15.084316 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:15.084316 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:15.085098 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:15.085098 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:15.085098 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:15.085098 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:15.085098 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 00:02:15.085098 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 00:02:15.085098 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 00:02:15.085098 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 00:02:15.085098 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 00:02:15.085098 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 00:02:15.085098 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 12 00:02:15.085098 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:15.085098 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 240/483] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Oct 12 00:02:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:15.085734 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fff764f-94e6-4eab-93db-787a74755462 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:15.086489 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5ec9a3-12a6-4fed-8094-baea456d7d85 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:15.122529 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:15.194008 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fff764f-94e6-4eab-93db-787a74755462 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:15.194201 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5ec9a3-12a6-4fed-8094-baea456d7d85 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:15.229575 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4385f7fe-c0f1-4b21-ae6e-5d022ebc4731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:15.230257 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4385f7fe-c0f1-4b21-ae6e-5d022ebc4731 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:15.230437 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab1e7e9-183a-4df6-bf2b-51ed96309364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:15.230547 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4385f7fe-c0f1-4b21-ae6e-5d022ebc4731 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:15.230797 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4385f7fe-c0f1-4b21-ae6e-5d022ebc4731 None admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:15.231003 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab1e7e9-183a-4df6-bf2b-51ed96309364 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:15.231279 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab1e7e9-183a-4df6-bf2b-51ed96309364 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:15.231502 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4385f7fe-c0f1-4b21-ae6e-5d022ebc4731 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:15.231593 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab1e7e9-183a-4df6-bf2b-51ed96309364 None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:15.231803 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4385f7fe-c0f1-4b21-ae6e-5d022ebc4731 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:15.232195 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab1e7e9-183a-4df6-bf2b-51ed96309364 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:15.232495 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab1e7e9-183a-4df6-bf2b-51ed96309364 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:15.233236 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4385f7fe-c0f1-4b21-ae6e-5d022ebc4731 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:15.233842 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab1e7e9-183a-4df6-bf2b-51ed96309364 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:15.255027 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 241/484] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Oct 12 00:02:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:15.255027 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4385f7fe-c0f1-4b21-ae6e-5d022ebc4731 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:15.255292 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 244/485] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Oct 12 00:02:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:15.255292 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab1e7e9-183a-4df6-bf2b-51ed96309364 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:15.303995 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:15.353505 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:15.360571 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab1e7e9-183a-4df6-bf2b-51ed96309364 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:15.361481 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4385f7fe-c0f1-4b21-ae6e-5d022ebc4731 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:15.398656 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fe6a88-c856-4eef-8efb-56df1d34b911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:15.399256 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fe6a88-c856-4eef-8efb-56df1d34b911 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:15.399574 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fe6a88-c856-4eef-8efb-56df1d34b911 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:15.399848 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fe6a88-c856-4eef-8efb-56df1d34b911 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:15.400573 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fe6a88-c856-4eef-8efb-56df1d34b911 None admin] RBAC: Adding query filter params (role.id=a53c0c4b8b8645b6a1cfe96266daec8e, scope.project.id=0697927c6aac43d4b14a750fbbb1145e, user.id=59b883abbb674b3c976bacb64194a66d) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:15.400688 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25c348a-e14b-4628-a108-0196dc208671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:15.400881 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fe6a88-c856-4eef-8efb-56df1d34b911 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:15.401246 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25c348a-e14b-4628-a108-0196dc208671 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:15.401524 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25c348a-e14b-4628-a108-0196dc208671 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:15.401795 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25c348a-e14b-4628-a108-0196dc208671 None admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:15.402315 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fe6a88-c856-4eef-8efb-56df1d34b911 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:15.402468 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25c348a-e14b-4628-a108-0196dc208671 None admin] RBAC: Adding query filter params (name=service) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:15.402740 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25c348a-e14b-4628-a108-0196dc208671 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:15.404246 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25c348a-e14b-4628-a108-0196dc208671 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:15.419756 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 242/486] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Oct 12 00:02:15 2022] GET /identity/v3/role_assignments?user.id=59b883abbb674b3c976bacb64194a66d&scope.project.id=0697927c6aac43d4b14a750fbbb1145e&role.id=a53c0c4b8b8645b6a1cfe96266daec8e => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:15.419756 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fe6a88-c856-4eef-8efb-56df1d34b911 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:15.429146 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 245/487] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Oct 12 00:02:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:15.430212 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25c348a-e14b-4628-a108-0196dc208671 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:15.569293 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25c348a-e14b-4628-a108-0196dc208671 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:15.576435 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fe6a88-c856-4eef-8efb-56df1d34b911 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:15.590333 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:15.607729 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c85692a-aad3-4e1d-9790-8f6f1d8940f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:15.608697 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c85692a-aad3-4e1d-9790-8f6f1d8940f1 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:15.609079 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c85692a-aad3-4e1d-9790-8f6f1d8940f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:15.609522 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c85692a-aad3-4e1d-9790-8f6f1d8940f1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:15.620058 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c85692a-aad3-4e1d-9790-8f6f1d8940f1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:15.621818 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c85692a-aad3-4e1d-9790-8f6f1d8940f1 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:15.626965 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29573945-459a-44f8-8421-80fc0e26fcf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:15.628483 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29573945-459a-44f8-8421-80fc0e26fcf2 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:15.628963 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29573945-459a-44f8-8421-80fc0e26fcf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:15.629417 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29573945-459a-44f8-8421-80fc0e26fcf2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:15.630647 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29573945-459a-44f8-8421-80fc0e26fcf2 None admin] RBAC: Adding query filter params (role.id=a53c0c4b8b8645b6a1cfe96266daec8e, scope.project.id=0697927c6aac43d4b14a750fbbb1145e, user.id=7f87febbbaa94626807a98da4a771ab1) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:15.631143 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29573945-459a-44f8-8421-80fc0e26fcf2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:15.632636 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29573945-459a-44f8-8421-80fc0e26fcf2 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:15.637046 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-5c85692a-aad3-4e1d-9790-8f6f1d8940f1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 00:02:15.637046 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:15.637046 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:15.637046 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:15.637046 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:15.637046 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:15.637046 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:15.637046 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:15.637046 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:15.637046 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:15.637046 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:15.637046 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:15.637046 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 00:02:15.637046 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 00:02:15.637046 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 00:02:15.637866 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 00:02:15.637866 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:15.637866 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:15.637866 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:15.637866 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:15.637866 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:15.637866 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:15.637866 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:15.637866 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:15.637866 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:15.637866 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:15.637866 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:15.637866 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:15.637866 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:15.637866 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:15.637866 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:15.637866 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:15.638696 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 00:02:15.638696 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 00:02:15.638696 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 00:02:15.638696 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 00:02:15.638696 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 00:02:15.638696 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 00:02:15.638696 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 00:02:15.638696 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:15.638696 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 246/488] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Oct 12 00:02:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:15.638696 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c85692a-aad3-4e1d-9790-8f6f1d8940f1 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:15.648244 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 243/489] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Oct 12 00:02:15 2022] GET /identity/v3/role_assignments?user.id=7f87febbbaa94626807a98da4a771ab1&scope.project.id=0697927c6aac43d4b14a750fbbb1145e&role.id=a53c0c4b8b8645b6a1cfe96266daec8e => generated 608 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:15.648406 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29573945-459a-44f8-8421-80fc0e26fcf2 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:15.659912 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:15.769681 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c85692a-aad3-4e1d-9790-8f6f1d8940f1 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:15.789072 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29573945-459a-44f8-8421-80fc0e26fcf2 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:15.814684 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf76f35-7a95-48fe-b584-5c82e0fee018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:15.816027 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf76f35-7a95-48fe-b584-5c82e0fee018 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:15.816309 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf76f35-7a95-48fe-b584-5c82e0fee018 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:15.816582 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf76f35-7a95-48fe-b584-5c82e0fee018 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:15.829874 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf76f35-7a95-48fe-b584-5c82e0fee018 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:15.831443 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf76f35-7a95-48fe-b584-5c82e0fee018 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:15.833479 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd32dd21-6e7d-4039-bd01-17d372989322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:15.834460 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd32dd21-6e7d-4039-bd01-17d372989322 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:15.834965 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd32dd21-6e7d-4039-bd01-17d372989322 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:15.835544 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd32dd21-6e7d-4039-bd01-17d372989322 None admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:15.836479 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd32dd21-6e7d-4039-bd01-17d372989322 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:15.836991 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd32dd21-6e7d-4039-bd01-17d372989322 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:15.838672 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd32dd21-6e7d-4039-bd01-17d372989322 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:15.849210 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-acf76f35-7a95-48fe-b584-5c82e0fee018 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:15.849210 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:15.849210 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:15.849210 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:15.849210 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:15.849210 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:15.849210 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:15.849210 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:15.849210 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:15.849210 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:15.849210 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:15.849210 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:15.849210 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 00:02:15.849210 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 00:02:15.849210 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 00:02:15.849210 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 00:02:15.850670 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:15.850670 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:15.850670 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:15.850670 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:15.850670 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:15.850670 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:15.850670 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:15.850670 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:15.850670 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:15.850670 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:15.850670 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:15.850670 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:15.850670 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:15.850670 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:15.850670 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:15.850670 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:15.850670 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 00:02:15.851769 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 00:02:15.851769 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:15.851769 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:15.856247 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 247/490] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:02:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:15.857424 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf76f35-7a95-48fe-b584-5c82e0fee018 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:15.863600 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 244/491] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Oct 12 00:02:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:15.866623 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd32dd21-6e7d-4039-bd01-17d372989322 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:15.905261 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:15.946380 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:15.992609 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf76f35-7a95-48fe-b584-5c82e0fee018 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:15.993425 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd32dd21-6e7d-4039-bd01-17d372989322 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:16.031226 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98ca2b6-778b-4617-8bc1-0f1fc8eac4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:16.031874 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98ca2b6-778b-4617-8bc1-0f1fc8eac4bf None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:16.032151 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98ca2b6-778b-4617-8bc1-0f1fc8eac4bf None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:16.032420 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98ca2b6-778b-4617-8bc1-0f1fc8eac4bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:16.033143 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98ca2b6-778b-4617-8bc1-0f1fc8eac4bf None admin] RBAC: Adding query filter params (role.id=a53c0c4b8b8645b6a1cfe96266daec8e, scope.project.id=0697927c6aac43d4b14a750fbbb1145e, user.id=51215dab618141b4991e232724094140) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:16.033446 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98ca2b6-778b-4617-8bc1-0f1fc8eac4bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:16.035448 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98ca2b6-778b-4617-8bc1-0f1fc8eac4bf None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:16.038567 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7750399-f418-4358-a163-458e939613c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:16.039429 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7750399-f418-4358-a163-458e939613c1 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:16.039937 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7750399-f418-4358-a163-458e939613c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:16.040397 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7750399-f418-4358-a163-458e939613c1 None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:16.041228 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7750399-f418-4358-a163-458e939613c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:16.041717 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7750399-f418-4358-a163-458e939613c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:16.041959 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 245/492] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Oct 12 00:02:15 2022] GET /identity/v3/role_assignments?user.id=51215dab618141b4991e232724094140&scope.project.id=0697927c6aac43d4b14a750fbbb1145e&role.id=a53c0c4b8b8645b6a1cfe96266daec8e => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 00:02:16.043149 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38d1739-eb7b-440d-969b-ff8de15c6a95 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:16.043453 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38d1739-eb7b-440d-969b-ff8de15c6a95 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:16.043727 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38d1739-eb7b-440d-969b-ff8de15c6a95 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:16.044253 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7750399-f418-4358-a163-458e939613c1 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:16.049014 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 246/493] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:02:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:02:16.051811 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a986581-5352-4235-b030-189cb408b4fc None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:16.052090 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a986581-5352-4235-b030-189cb408b4fc None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:16.052362 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a986581-5352-4235-b030-189cb408b4fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:16.073654 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 248/494] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Oct 12 00:02:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:16.074746 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-4a986581-5352-4235-b030-189cb408b4fc None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:02:16.077551 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-4a986581-5352-4235-b030-189cb408b4fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:02:16.079965 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7750399-f418-4358-a163-458e939613c1 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:16.154169 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 247/495] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:02:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 12 00:02:16.271082 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7750399-f418-4358-a163-458e939613c1 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:16.300320 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:16.308745 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbe9c71-b3cc-4a1f-9b34-b5b578bd380f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:16.309382 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbe9c71-b3cc-4a1f-9b34-b5b578bd380f None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:16.309655 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbe9c71-b3cc-4a1f-9b34-b5b578bd380f None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:16.309921 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbe9c71-b3cc-4a1f-9b34-b5b578bd380f None admin] PATH_INFO: `/v3/users/swift` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:16.317634 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbe9c71-b3cc-4a1f-9b34-b5b578bd380f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:16.319213 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbe9c71-b3cc-4a1f-9b34-b5b578bd380f None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:16.331561 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-5fbe9c71-b3cc-4a1f-9b34-b5b578bd380f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 12 00:02:16.331561 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:16.331561 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:16.331561 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:16.331561 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:16.331561 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:16.331561 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:16.331561 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:16.331561 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:16.331561 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:16.331561 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:16.331561 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:16.331561 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 00:02:16.331561 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 00:02:16.331561 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 00:02:16.331561 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 00:02:16.332766 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:16.332766 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:16.332766 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 00:02:16.332766 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 00:02:16.332766 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 00:02:16.332766 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 00:02:16.332766 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:16.332766 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:16.332766 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:16.332766 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:16.332766 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:16.332766 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:16.332766 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:16.332766 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:16.332766 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:16.332766 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:16.332766 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:16.332766 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:16.333929 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:16.333929 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:16.333929 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 00:02:16.333929 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 00:02:16.333929 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 00:02:16.333929 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 00:02:16.333929 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 00:02:16.333929 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 00:02:16.333929 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 12 00:02:16.333929 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:16.333929 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 249/496] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Oct 12 00:02:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:16.338288 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a986581-5352-4235-b030-189cb408b4fc None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:16.368962 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:16.447903 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a986581-5352-4235-b030-189cb408b4fc None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:16.483666 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ace5154-a1ca-4f15-9643-ed5b3c73cd90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:16.484518 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ace5154-a1ca-4f15-9643-ed5b3c73cd90 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:16.484942 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ace5154-a1ca-4f15-9643-ed5b3c73cd90 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:16.485358 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ace5154-a1ca-4f15-9643-ed5b3c73cd90 None admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:16.486309 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ace5154-a1ca-4f15-9643-ed5b3c73cd90 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:16.486754 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ace5154-a1ca-4f15-9643-ed5b3c73cd90 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:16.488853 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ace5154-a1ca-4f15-9643-ed5b3c73cd90 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:16.508653 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbe9c71-b3cc-4a1f-9b34-b5b578bd380f None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:16.512546 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 248/497] 10.222.0.36 () {56 vars in 1135 bytes} [Wed Oct 12 00:02:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:16.518716 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ace5154-a1ca-4f15-9643-ed5b3c73cd90 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:16.645712 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbe9c71-b3cc-4a1f-9b34-b5b578bd380f None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:16.685347 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781381cb-cfa4-4510-8cff-15771092c5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:16.686402 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781381cb-cfa4-4510-8cff-15771092c5d2 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:16.686903 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781381cb-cfa4-4510-8cff-15771092c5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:16.687394 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781381cb-cfa4-4510-8cff-15771092c5d2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:16.699750 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781381cb-cfa4-4510-8cff-15771092c5d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 00:02:16.707562 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-781381cb-cfa4-4510-8cff-15771092c5d2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 00:02:16.707562 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:16.707562 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:16.707562 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:16.707562 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:16.707562 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:16.707562 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:16.707562 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:16.707562 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:16.707562 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:16.707562 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:16.707562 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:16.707562 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 00:02:16.707562 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 00:02:16.707562 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 00:02:16.707562 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 00:02:16.708553 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 00:02:16.708553 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 00:02:16.708553 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 00:02:16.708553 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 00:02:16.708553 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:16.708553 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:16.708553 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:16.708553 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:16.708553 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:16.708553 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:16.708553 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:16.708553 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:16.708553 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:16.708553 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:16.708553 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:16.708553 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:16.709349 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:16.709349 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:16.709349 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:16.709349 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:16.709349 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 00:02:16.709349 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 00:02:16.709349 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 00:02:16.709349 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 00:02:16.709349 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 00:02:16.709349 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 00:02:16.709349 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 00:02:16.709349 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:16.712328 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 250/498] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Oct 12 00:02:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:16.717823 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781381cb-cfa4-4510-8cff-15771092c5d2 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:16.733554 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:16.772653 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ace5154-a1ca-4f15-9643-ed5b3c73cd90 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:16.834144 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:16.857003 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751166fa-96fd-4839-8fad-304b88c5172f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:16.857625 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751166fa-96fd-4839-8fad-304b88c5172f None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:16.857904 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751166fa-96fd-4839-8fad-304b88c5172f None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:16.858235 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751166fa-96fd-4839-8fad-304b88c5172f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:16.894956 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781381cb-cfa4-4510-8cff-15771092c5d2 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:16.934715 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1431d4f3-bf54-4dfa-ab57-1a6e0a06a6c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:16.935724 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1431d4f3-bf54-4dfa-ab57-1a6e0a06a6c9 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:16.936204 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1431d4f3-bf54-4dfa-ab57-1a6e0a06a6c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:16.936673 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1431d4f3-bf54-4dfa-ab57-1a6e0a06a6c9 None admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:16.937813 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1431d4f3-bf54-4dfa-ab57-1a6e0a06a6c9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:16.938693 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1431d4f3-bf54-4dfa-ab57-1a6e0a06a6c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:16.940603 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1431d4f3-bf54-4dfa-ab57-1a6e0a06a6c9 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:16.950915 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 251/499] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Oct 12 00:02:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:16.960508 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1431d4f3-bf54-4dfa-ab57-1a6e0a06a6c9 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:17.032384 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751166fa-96fd-4839-8fad-304b88c5172f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:17.034404 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751166fa-96fd-4839-8fad-304b88c5172f None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:17.070844 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1431d4f3-bf54-4dfa-ab57-1a6e0a06a6c9 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:17.086970 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:17.115922 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a98c0e3-c9aa-4b46-a295-3f74747d80ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:17.117483 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a98c0e3-c9aa-4b46-a295-3f74747d80ad None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:17.117966 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a98c0e3-c9aa-4b46-a295-3f74747d80ad None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:17.118449 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a98c0e3-c9aa-4b46-a295-3f74747d80ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:17.131307 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a98c0e3-c9aa-4b46-a295-3f74747d80ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:17.134180 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a98c0e3-c9aa-4b46-a295-3f74747d80ad None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:17.151044 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-8a98c0e3-c9aa-4b46-a295-3f74747d80ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:17.151044 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:17.151044 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:17.151044 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:17.151044 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:17.151044 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:17.151044 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:17.151044 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:17.151044 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:17.151044 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:17.151044 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:17.151044 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:17.151044 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 00:02:17.151044 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 00:02:17.151044 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 00:02:17.151044 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 00:02:17.152492 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:17.152492 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:17.152492 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:17.152492 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:17.152492 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:17.152492 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:17.152492 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:17.152492 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:17.152492 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:17.152492 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:17.152492 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:17.152492 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:17.152492 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:17.152492 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:17.152492 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:17.152492 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:17.152492 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 00:02:17.153365 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 00:02:17.153365 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:17.153365 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:17.155759 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 252/500] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:02:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:17.163410 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a98c0e3-c9aa-4b46-a295-3f74747d80ad None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:17.233672 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:17.262475 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 249/501] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Oct 12 00:02:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 745 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 12 00:02:17.269781 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751166fa-96fd-4839-8fad-304b88c5172f None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:17.352031 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a98c0e3-c9aa-4b46-a295-3f74747d80ad None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:17.390084 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92dfc10-231a-4b69-b43a-444e3a06f655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:17.390692 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92dfc10-231a-4b69-b43a-444e3a06f655 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:17.390973 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92dfc10-231a-4b69-b43a-444e3a06f655 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:17.391244 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92dfc10-231a-4b69-b43a-444e3a06f655 None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:17.391909 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92dfc10-231a-4b69-b43a-444e3a06f655 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:17.392207 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92dfc10-231a-4b69-b43a-444e3a06f655 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:17.394501 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92dfc10-231a-4b69-b43a-444e3a06f655 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:17.407288 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 253/502] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Oct 12 00:02:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:17.413606 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92dfc10-231a-4b69-b43a-444e3a06f655 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:17.439133 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:17.461211 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751166fa-96fd-4839-8fad-304b88c5172f None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:17.525744 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c41b4a-a44a-4e13-9e67-b432b583f2d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:17.526294 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c41b4a-a44a-4e13-9e67-b432b583f2d0 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:17.526568 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c41b4a-a44a-4e13-9e67-b432b583f2d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:17.526834 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c41b4a-a44a-4e13-9e67-b432b583f2d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:17.549651 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c41b4a-a44a-4e13-9e67-b432b583f2d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:17.550831 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c41b4a-a44a-4e13-9e67-b432b583f2d0 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:17.578620 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-a9c41b4a-a44a-4e13-9e67-b432b583f2d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:17.578620 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:17.578620 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:17.578620 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:17.578620 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:17.578620 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:17.578620 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:17.578620 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:17.578620 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:17.578620 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:17.578620 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:17.578620 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:17.578620 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 00:02:17.578620 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 00:02:17.578620 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 00:02:17.578620 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 00:02:17.579731 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:17.579731 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:17.579731 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:17.579731 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:17.579731 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:17.579731 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:17.579731 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:17.579731 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:17.579731 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:17.579731 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:17.579731 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:17.579731 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:17.579731 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:17.579731 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:17.579731 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:17.579731 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:17.579731 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 00:02:17.580729 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 00:02:17.580729 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:17.580729 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:17.587720 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 250/503] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:02:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:17.596334 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92dfc10-231a-4b69-b43a-444e3a06f655 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:17.596628 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c41b4a-a44a-4e13-9e67-b432b583f2d0 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:17.641407 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bdb4f0-ee2b-4ce6-8e3d-637cbd62111b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:17.642094 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bdb4f0-ee2b-4ce6-8e3d-637cbd62111b None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:17.642371 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bdb4f0-ee2b-4ce6-8e3d-637cbd62111b None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:17.642636 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bdb4f0-ee2b-4ce6-8e3d-637cbd62111b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:17.656652 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bdb4f0-ee2b-4ce6-8e3d-637cbd62111b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:17.658327 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bdb4f0-ee2b-4ce6-8e3d-637cbd62111b None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:17.665343 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:17.677277 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-25bdb4f0-ee2b-4ce6-8e3d-637cbd62111b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 12 00:02:17.677277 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:17.677277 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:17.677277 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:17.677277 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:17.677277 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:17.677277 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:17.677277 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:17.677277 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:17.677277 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:17.677277 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:17.677277 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:17.677277 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 00:02:17.677277 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 00:02:17.677277 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 00:02:17.677277 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 00:02:17.692697 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:17.692697 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:17.692697 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 00:02:17.692697 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 00:02:17.692697 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 00:02:17.692697 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 00:02:17.692697 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:17.692697 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:17.692697 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:17.692697 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:17.692697 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:17.692697 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:17.692697 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:17.692697 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:17.692697 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:17.692697 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:17.692697 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:17.692697 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:17.693624 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:17.693624 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:17.693624 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 00:02:17.693624 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 00:02:17.693624 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 00:02:17.693624 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 00:02:17.693624 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 00:02:17.693624 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 00:02:17.693624 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 12 00:02:17.693624 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:17.693624 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 254/504] 10.222.0.36 () {56 vars in 1093 bytes} [Wed Oct 12 00:02:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 00:02:17.693624 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bdb4f0-ee2b-4ce6-8e3d-637cbd62111b None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:17.777135 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c41b4a-a44a-4e13-9e67-b432b583f2d0 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:17.831392 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91703f9b-45ba-4afa-8d55-a69552be2721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:17.831392 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91703f9b-45ba-4afa-8d55-a69552be2721 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:17.831392 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91703f9b-45ba-4afa-8d55-a69552be2721 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:17.831392 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91703f9b-45ba-4afa-8d55-a69552be2721 None admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:17.831831 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91703f9b-45ba-4afa-8d55-a69552be2721 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:17.831831 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91703f9b-45ba-4afa-8d55-a69552be2721 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:17.831831 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91703f9b-45ba-4afa-8d55-a69552be2721 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:17.851396 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 251/505] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Oct 12 00:02:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:17.851396 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91703f9b-45ba-4afa-8d55-a69552be2721 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:17.868465 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:17.905941 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bdb4f0-ee2b-4ce6-8e3d-637cbd62111b None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:17.952194 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc5c26d-0b5a-466e-b9f0-64fc51b0daad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:17.952798 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc5c26d-0b5a-466e-b9f0-64fc51b0daad None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:17.953073 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc5c26d-0b5a-466e-b9f0-64fc51b0daad None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:17.953344 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc5c26d-0b5a-466e-b9f0-64fc51b0daad None admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:17.954070 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc5c26d-0b5a-466e-b9f0-64fc51b0daad None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:17.954378 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc5c26d-0b5a-466e-b9f0-64fc51b0daad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:17.955898 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc5c26d-0b5a-466e-b9f0-64fc51b0daad None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:17.973781 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 255/506] 10.222.0.36 () {56 vars in 1133 bytes} [Wed Oct 12 00:02:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:17.979731 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc5c26d-0b5a-466e-b9f0-64fc51b0daad None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:18.007138 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:18.018702 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91703f9b-45ba-4afa-8d55-a69552be2721 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:18.081749 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e05b931-1adb-40a4-a2db-f60bdafd6f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:18.082389 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e05b931-1adb-40a4-a2db-f60bdafd6f35 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:18.082669 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e05b931-1adb-40a4-a2db-f60bdafd6f35 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:18.082994 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e05b931-1adb-40a4-a2db-f60bdafd6f35 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:18.091518 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e05b931-1adb-40a4-a2db-f60bdafd6f35 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:18.093668 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e05b931-1adb-40a4-a2db-f60bdafd6f35 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:18.109321 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-7e05b931-1adb-40a4-a2db-f60bdafd6f35 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 00:02:18.109321 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:18.109321 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:18.109321 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:18.109321 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:18.109321 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:18.109321 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:18.109321 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:18.109321 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:18.109321 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:18.109321 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:18.109321 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:18.109321 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 00:02:18.109321 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 00:02:18.109321 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 00:02:18.110216 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 00:02:18.110216 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:18.110216 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:18.110216 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:18.110216 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:18.110216 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:18.110216 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:18.110216 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:18.110216 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:18.110216 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:18.110216 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:18.110216 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:18.110216 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:18.110216 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:18.110216 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:18.110216 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:18.110216 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:18.111085 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 00:02:18.111085 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 00:02:18.111085 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 00:02:18.111085 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 00:02:18.111085 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 00:02:18.111085 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 00:02:18.111085 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 00:02:18.111085 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:18.111085 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 252/507] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Oct 12 00:02:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:18.118476 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e05b931-1adb-40a4-a2db-f60bdafd6f35 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:18.171230 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc5c26d-0b5a-466e-b9f0-64fc51b0daad None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:18.218119 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd81c3a3-dbd0-4d62-9742-45ae40d46adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:18.218757 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd81c3a3-dbd0-4d62-9742-45ae40d46adb None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:18.219128 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd81c3a3-dbd0-4d62-9742-45ae40d46adb None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:18.219421 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd81c3a3-dbd0-4d62-9742-45ae40d46adb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:18.228703 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:18.231641 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd81c3a3-dbd0-4d62-9742-45ae40d46adb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:18.233100 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd81c3a3-dbd0-4d62-9742-45ae40d46adb None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:18.249034 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-fd81c3a3-dbd0-4d62-9742-45ae40d46adb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:18.249034 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:18.249034 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:18.249034 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:18.249034 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:18.249034 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:18.249034 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:18.249034 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:18.249034 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:18.249034 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:18.249034 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:18.249034 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:18.249034 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 00:02:18.249034 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 00:02:18.249034 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 00:02:18.249034 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 00:02:18.250008 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:18.250008 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:18.250008 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:18.250008 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:18.250008 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:18.250008 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:18.250008 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:18.250008 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:18.250008 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:18.250008 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:18.250008 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:18.250008 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:18.250008 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:18.250008 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:18.250008 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:18.250008 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:18.250008 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 00:02:18.250874 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 00:02:18.250874 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:18.250874 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:18.250874 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 256/508] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:02:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 00:02:18.252501 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bfe6ac-36a9-4d73-beaf-222078df4867 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:18.252501 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bfe6ac-36a9-4d73-beaf-222078df4867 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:18.252501 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bfe6ac-36a9-4d73-beaf-222078df4867 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:18.256533 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 257/509] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:02:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:02:18.257553 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bfe6ac-36a9-4d73-beaf-222078df4867 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:18.259799 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e05b931-1adb-40a4-a2db-f60bdafd6f35 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:18.295325 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244844dc-5d58-4855-9381-3daf2da009b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:18.295961 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244844dc-5d58-4855-9381-3daf2da009b3 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:18.296239 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244844dc-5d58-4855-9381-3daf2da009b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:18.296507 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244844dc-5d58-4855-9381-3daf2da009b3 None admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:18.297193 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244844dc-5d58-4855-9381-3daf2da009b3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:18.297493 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244844dc-5d58-4855-9381-3daf2da009b3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:18.298933 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244844dc-5d58-4855-9381-3daf2da009b3 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:18.311394 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 253/510] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Oct 12 00:02:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:18.312634 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c147e672-45b2-476f-bf1f-5e45c2a34d26 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:18.312879 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c147e672-45b2-476f-bf1f-5e45c2a34d26 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:18.313145 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c147e672-45b2-476f-bf1f-5e45c2a34d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:18.330249 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-c147e672-45b2-476f-bf1f-5e45c2a34d26 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:02:18.331488 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-c147e672-45b2-476f-bf1f-5e45c2a34d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:02:18.342126 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:18.375621 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bfe6ac-36a9-4d73-beaf-222078df4867 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:18.409452 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ede3483-96ff-4c2a-a19e-7465c30aa2d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:18.410061 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ede3483-96ff-4c2a-a19e-7465c30aa2d9 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:18.410348 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ede3483-96ff-4c2a-a19e-7465c30aa2d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:18.410600 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ede3483-96ff-4c2a-a19e-7465c30aa2d9 None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:18.411228 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ede3483-96ff-4c2a-a19e-7465c30aa2d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:18.411561 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ede3483-96ff-4c2a-a19e-7465c30aa2d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:18.411729 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 254/511] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:02:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 00:02:18.413056 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ede3483-96ff-4c2a-a19e-7465c30aa2d9 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:18.413332 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d7083-0550-436f-ba18-b9dc64a1889e None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:18.413622 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d7083-0550-436f-ba18-b9dc64a1889e None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:18.413895 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d7083-0550-436f-ba18-b9dc64a1889e None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:18.415432 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 255/512] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:02:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:02:18.419598 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39d7083-0550-436f-ba18-b9dc64a1889e None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:18.429495 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 258/513] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Oct 12 00:02:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:18.430934 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67bd563-20ce-4b0f-a3fa-9bf4aa9e9f5a None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:18.431347 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67bd563-20ce-4b0f-a3fa-9bf4aa9e9f5a None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:18.431769 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67bd563-20ce-4b0f-a3fa-9bf4aa9e9f5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:18.450632 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-b67bd563-20ce-4b0f-a3fa-9bf4aa9e9f5a None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:02:18.452149 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-b67bd563-20ce-4b0f-a3fa-9bf4aa9e9f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:02:18.519814 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:18.527692 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 259/514] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:02:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 00:02:18.529649 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67bd563-20ce-4b0f-a3fa-9bf4aa9e9f5a None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:18.542580 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39d7083-0550-436f-ba18-b9dc64a1889e None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:18.608688 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b878c7-47e2-49cc-8b9a-c0fa0898339c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:18.610402 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b878c7-47e2-49cc-8b9a-c0fa0898339c None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:18.610672 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b878c7-47e2-49cc-8b9a-c0fa0898339c None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:18.610942 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b878c7-47e2-49cc-8b9a-c0fa0898339c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:18.612996 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b878c7-47e2-49cc-8b9a-c0fa0898339c None admin] RBAC: Adding query filter params (role.id=a53c0c4b8b8645b6a1cfe96266daec8e, scope.project.id=0697927c6aac43d4b14a750fbbb1145e, user.id=fc1bcffc78844dcbb1a9b7c80285249b) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:18.615254 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b878c7-47e2-49cc-8b9a-c0fa0898339c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:18.618907 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b878c7-47e2-49cc-8b9a-c0fa0898339c None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:18.630638 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 256/515] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Oct 12 00:02:18 2022] GET /identity/v3/role_assignments?user.id=fc1bcffc78844dcbb1a9b7c80285249b&scope.project.id=0697927c6aac43d4b14a750fbbb1145e&role.id=a53c0c4b8b8645b6a1cfe96266daec8e => generated 608 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:18.687040 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b878c7-47e2-49cc-8b9a-c0fa0898339c None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:18.701450 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:18.748262 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67bd563-20ce-4b0f-a3fa-9bf4aa9e9f5a None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:18.799207 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c545172-2bea-4ddf-845b-bb75e318b854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:18.801106 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c545172-2bea-4ddf-845b-bb75e318b854 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:18.801856 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c545172-2bea-4ddf-845b-bb75e318b854 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:18.802372 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c545172-2bea-4ddf-845b-bb75e318b854 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:18.812673 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c545172-2bea-4ddf-845b-bb75e318b854 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:18.815918 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c545172-2bea-4ddf-845b-bb75e318b854 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:18.820126 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b878c7-47e2-49cc-8b9a-c0fa0898339c None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:18.831504 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-5c545172-2bea-4ddf-845b-bb75e318b854 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 00:02:18.831504 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:18.831504 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:18.831504 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:18.831504 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:18.831504 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:18.831504 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:18.831504 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:18.831504 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:18.831504 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:18.831504 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:18.831504 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:18.831504 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 00:02:18.831504 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 00:02:18.831504 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 00:02:18.832517 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 00:02:18.832517 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:18.832517 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:18.832517 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:18.832517 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:18.832517 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:18.832517 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:18.832517 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:18.832517 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:18.832517 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:18.832517 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:18.832517 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:18.832517 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:18.832517 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:18.832517 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:18.832517 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:18.832517 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:18.833458 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 00:02:18.833458 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 00:02:18.833458 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 00:02:18.833458 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 00:02:18.833458 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 00:02:18.833458 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 00:02:18.833458 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 00:02:18.833458 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:18.833877 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 260/516] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Oct 12 00:02:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:18.839550 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b561b4ae-c142-4314-a731-752ae86b5301 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:18.839550 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b561b4ae-c142-4314-a731-752ae86b5301 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:18.839550 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b561b4ae-c142-4314-a731-752ae86b5301 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:18.839550 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 261/517] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:02:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:02:18.843004 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b561b4ae-c142-4314-a731-752ae86b5301 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:18.872422 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcae0d2-b52e-44c5-825e-4b8ad0d5ae99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:18.873012 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcae0d2-b52e-44c5-825e-4b8ad0d5ae99 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:18.873289 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcae0d2-b52e-44c5-825e-4b8ad0d5ae99 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:18.873559 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcae0d2-b52e-44c5-825e-4b8ad0d5ae99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:18.886751 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcae0d2-b52e-44c5-825e-4b8ad0d5ae99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:18.888236 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcae0d2-b52e-44c5-825e-4b8ad0d5ae99 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:18.893315 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:18.911678 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-2dcae0d2-b52e-44c5-825e-4b8ad0d5ae99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:18.911678 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:18.911678 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:18.911678 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:18.911678 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:18.911678 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:18.911678 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:18.911678 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:18.911678 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:18.911678 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:18.911678 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:18.911678 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:18.911678 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 00:02:18.911678 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 00:02:18.911678 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 00:02:18.911678 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 00:02:18.921680 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:18.921680 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:18.921680 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:18.921680 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:18.921680 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:18.921680 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:18.921680 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:18.921680 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:18.921680 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:18.921680 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:18.921680 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:18.921680 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:18.921680 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:18.921680 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:18.921680 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:18.921680 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:18.921680 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 00:02:18.922578 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 00:02:18.922578 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:18.922578 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:18.922578 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 257/518] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:02:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 00:02:18.922578 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c7cc4a-3393-46e0-8d4c-b40e60894b19 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:18.922578 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c7cc4a-3393-46e0-8d4c-b40e60894b19 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:18.922578 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c7cc4a-3393-46e0-8d4c-b40e60894b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:18.937326 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-a4c7cc4a-3393-46e0-8d4c-b40e60894b19 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:02:18.938803 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-a4c7cc4a-3393-46e0-8d4c-b40e60894b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:02:18.962731 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b561b4ae-c142-4314-a731-752ae86b5301 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:18.999114 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:19.000251 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e119fe3-b92c-43fc-99d5-6422ed5cc53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:19.000878 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e119fe3-b92c-43fc-99d5-6422ed5cc53e None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:19.001184 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e119fe3-b92c-43fc-99d5-6422ed5cc53e None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:19.001583 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e119fe3-b92c-43fc-99d5-6422ed5cc53e None admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:19.002466 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e119fe3-b92c-43fc-99d5-6422ed5cc53e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:19.002573 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e119fe3-b92c-43fc-99d5-6422ed5cc53e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:19.004036 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e119fe3-b92c-43fc-99d5-6422ed5cc53e None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:19.019392 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 258/519] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:02:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 00:02:19.019392 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c7cc4a-3393-46e0-8d4c-b40e60894b19 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:19.027023 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 262/520] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Oct 12 00:02:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:19.028223 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e119fe3-b92c-43fc-99d5-6422ed5cc53e None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:19.136797 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c7cc4a-3393-46e0-8d4c-b40e60894b19 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:19.152301 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e119fe3-b92c-43fc-99d5-6422ed5cc53e None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:19.154280 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:19.173564 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f1013e-7412-4e1c-9774-bda0f0bddc88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:19.174206 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f1013e-7412-4e1c-9774-bda0f0bddc88 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:19.175025 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f1013e-7412-4e1c-9774-bda0f0bddc88 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:19.175296 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f1013e-7412-4e1c-9774-bda0f0bddc88 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:19.175915 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f1013e-7412-4e1c-9774-bda0f0bddc88 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:19.177480 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f1013e-7412-4e1c-9774-bda0f0bddc88 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:19.191603 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-91f1013e-7412-4e1c-9774-bda0f0bddc88 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 12 00:02:19.191603 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:19.191603 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 00:02:19.191603 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 00:02:19.191603 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 00:02:19.191603 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 00:02:19.191603 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 00:02:19.191603 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 00:02:19.191603 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 12 00:02:19.191603 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:19.191603 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 00:02:19.191603 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:19.191603 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:19.191603 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:19.191603 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:19.191603 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:19.191603 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:19.192620 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:19.192620 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:19.192620 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:19.192620 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:19.192620 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:19.192620 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:19.192620 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 00:02:19.192620 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 00:02:19.192620 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 00:02:19.192620 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 00:02:19.192620 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:19.192620 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:19.192620 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:19.192620 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:19.192620 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:19.192620 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:19.193457 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:19.193457 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:19.193457 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:19.193457 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:19.193457 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:19.193457 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:19.193457 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:19.193457 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:19.193457 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:19.193457 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:19.193457 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 00:02:19.193457 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 00:02:19.193457 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 12 00:02:19.193457 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:19.195586 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 259/521] 10.222.0.36 () {56 vars in 1129 bytes} [Wed Oct 12 00:02:19 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:19.196812 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f1013e-7412-4e1c-9774-bda0f0bddc88 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:19.224383 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9590f65-be51-4b60-8800-93f261417d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:19.225120 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9590f65-be51-4b60-8800-93f261417d34 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:19.225435 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9590f65-be51-4b60-8800-93f261417d34 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:19.225735 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9590f65-be51-4b60-8800-93f261417d34 None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:19.226415 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9590f65-be51-4b60-8800-93f261417d34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:19.226745 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9590f65-be51-4b60-8800-93f261417d34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:19.228913 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9590f65-be51-4b60-8800-93f261417d34 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:19.247172 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 263/522] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Oct 12 00:02:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:19.247172 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9590f65-be51-4b60-8800-93f261417d34 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:19.294325 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:19.343969 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f1013e-7412-4e1c-9774-bda0f0bddc88 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:19.378449 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58d68f3-b0ed-43a3-b345-9fcdaf8695db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:19.379108 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58d68f3-b0ed-43a3-b345-9fcdaf8695db None admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:19.379420 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58d68f3-b0ed-43a3-b345-9fcdaf8695db None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:19.379694 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58d68f3-b0ed-43a3-b345-9fcdaf8695db None admin] PATH_INFO: `/v3/projects/0697927c6aac43d4b14a750fbbb1145e/users/a515b75aa3944912ad2364ffd7924f40/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:19.381631 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58d68f3-b0ed-43a3-b345-9fcdaf8695db None admin] RBAC: Authorizing `identity:create_grant(project_id=0697927c6aac43d4b14a750fbbb1145e, user_id=a515b75aa3944912ad2364ffd7924f40, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:19.383094 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58d68f3-b0ed-43a3-b345-9fcdaf8695db None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:19.395995 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9590f65-be51-4b60-8800-93f261417d34 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:19.432249 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af29b9f-ddf3-4529-8354-7d553a5c8f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:19.432860 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af29b9f-ddf3-4529-8354-7d553a5c8f51 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:19.433138 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af29b9f-ddf3-4529-8354-7d553a5c8f51 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:19.433405 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af29b9f-ddf3-4529-8354-7d553a5c8f51 None admin] PATH_INFO: `/v3/services` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:19.434026 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af29b9f-ddf3-4529-8354-7d553a5c8f51 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:19.434322 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af29b9f-ddf3-4529-8354-7d553a5c8f51 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:19.435775 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af29b9f-ddf3-4529-8354-7d553a5c8f51 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:19.441202 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 264/523] 10.222.0.36 () {56 vars in 1124 bytes} [Wed Oct 12 00:02:19 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:19.442019 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af29b9f-ddf3-4529-8354-7d553a5c8f51 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:19.460663 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 260/524] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Oct 12 00:02:19 2022] PUT /identity/v3/projects/0697927c6aac43d4b14a750fbbb1145e/users/a515b75aa3944912ad2364ffd7924f40/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:02:19.461501 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58d68f3-b0ed-43a3-b345-9fcdaf8695db None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:19.507521 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:19.613408 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58d68f3-b0ed-43a3-b345-9fcdaf8695db None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:19.637805 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:19.665823 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aa08f6-3d33-4e07-9245-7b1c139c3b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:19.666968 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aa08f6-3d33-4e07-9245-7b1c139c3b58 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:19.668369 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aa08f6-3d33-4e07-9245-7b1c139c3b58 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:19.668828 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aa08f6-3d33-4e07-9245-7b1c139c3b58 None admin] PATH_INFO: `/v3/services/network` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:19.669765 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aa08f6-3d33-4e07-9245-7b1c139c3b58 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:19.671849 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aa08f6-3d33-4e07-9245-7b1c139c3b58 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:19.679903 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af29b9f-ddf3-4529-8354-7d553a5c8f51 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:19.685332 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-a5aa08f6-3d33-4e07-9245-7b1c139c3b58 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 12 00:02:19.685332 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:19.685332 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 00:02:19.685332 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 00:02:19.685332 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 00:02:19.685332 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 00:02:19.685332 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 00:02:19.685332 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 00:02:19.685332 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 12 00:02:19.685332 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:19.685332 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 00:02:19.685332 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:19.685332 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:19.685332 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:19.685332 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:19.685332 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:19.685332 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:19.686312 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:19.686312 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:19.686312 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:19.686312 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:19.686312 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:19.686312 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:19.686312 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 00:02:19.686312 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 00:02:19.686312 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 00:02:19.686312 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 00:02:19.686312 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:19.686312 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:19.686312 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:19.686312 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:19.686312 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:19.686312 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:19.687123 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:19.687123 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:19.687123 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:19.687123 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:19.687123 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:19.687123 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:19.687123 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:19.687123 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:19.687123 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:19.687123 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:19.687123 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 00:02:19.687123 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 00:02:19.687123 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 12 00:02:19.687123 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:19.689674 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 261/525] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Oct 12 00:02:19 2022] GET /identity/v3/services/network => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:19.691378 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aa08f6-3d33-4e07-9245-7b1c139c3b58 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:19.739783 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b790c0-3ae4-46aa-a1aa-5aec477b4dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:19.740389 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b790c0-3ae4-46aa-a1aa-5aec477b4dab None admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:19.740664 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b790c0-3ae4-46aa-a1aa-5aec477b4dab None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:19.740935 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b790c0-3ae4-46aa-a1aa-5aec477b4dab None admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:19.741577 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b790c0-3ae4-46aa-a1aa-5aec477b4dab None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:19.743034 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b790c0-3ae4-46aa-a1aa-5aec477b4dab None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:19.825104 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aa08f6-3d33-4e07-9245-7b1c139c3b58 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:19.831551 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 265/526] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Oct 12 00:02:19 2022] POST /identity/v3/users => generated 254 bytes in 382 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:02:19.832606 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b790c0-3ae4-46aa-a1aa-5aec477b4dab None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:19.873650 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0df3dc-d5ef-4b10-b5da-caffd66154c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:19.874639 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0df3dc-d5ef-4b10-b5da-caffd66154c7 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:19.875209 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0df3dc-d5ef-4b10-b5da-caffd66154c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:19.875713 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0df3dc-d5ef-4b10-b5da-caffd66154c7 None admin] PATH_INFO: `/v3/services` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:19.876701 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0df3dc-d5ef-4b10-b5da-caffd66154c7 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:19.878061 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0df3dc-d5ef-4b10-b5da-caffd66154c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:19.880158 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0df3dc-d5ef-4b10-b5da-caffd66154c7 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:19.895972 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 262/527] 10.222.0.36 () {56 vars in 1124 bytes} [Wed Oct 12 00:02:19 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:19.988008 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:20.019881 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b790c0-3ae4-46aa-a1aa-5aec477b4dab None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:20.070687 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af3e6dc-ef28-43de-9ec6-75cf6563ee50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:20.071323 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af3e6dc-ef28-43de-9ec6-75cf6563ee50 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:20.071621 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af3e6dc-ef28-43de-9ec6-75cf6563ee50 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:20.071886 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af3e6dc-ef28-43de-9ec6-75cf6563ee50 None admin] PATH_INFO: `/v3/services` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:20.072525 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af3e6dc-ef28-43de-9ec6-75cf6563ee50 None admin] RBAC: Adding query filter params (name=network) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:20.072819 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af3e6dc-ef28-43de-9ec6-75cf6563ee50 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:20.074195 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af3e6dc-ef28-43de-9ec6-75cf6563ee50 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:20.084470 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 266/528] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Oct 12 00:02:19 2022] GET /identity/v3/services?name=network => generated 124 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:20.090453 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0df3dc-d5ef-4b10-b5da-caffd66154c7 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:20.164506 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:20.223983 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0df3dc-d5ef-4b10-b5da-caffd66154c7 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:20.260288 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d019411-3883-4beb-be45-d3dd9a9b7e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:20.260814 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d019411-3883-4beb-be45-d3dd9a9b7e2f None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:20.261093 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d019411-3883-4beb-be45-d3dd9a9b7e2f None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:20.261532 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d019411-3883-4beb-be45-d3dd9a9b7e2f None admin] PATH_INFO: `/v3/services` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:20.262167 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d019411-3883-4beb-be45-d3dd9a9b7e2f None admin] RBAC: Adding query filter params (type=network) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:20.262505 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d019411-3883-4beb-be45-d3dd9a9b7e2f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:20.263916 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d019411-3883-4beb-be45-d3dd9a9b7e2f None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:20.269525 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 263/529] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Oct 12 00:02:20 2022] GET /identity/v3/services?type=network => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:20.936429 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdca4ee-8b4d-422d-9a6f-ad29616979eb None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:20.937434 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdca4ee-8b4d-422d-9a6f-ad29616979eb None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:20.937910 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdca4ee-8b4d-422d-9a6f-ad29616979eb None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:20.939503 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 267/530] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:02:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:02:20.945483 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1740b91-b784-48be-b364-349e7f7b5dd9 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:20.945898 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1740b91-b784-48be-b364-349e7f7b5dd9 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:20.946083 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1740b91-b784-48be-b364-349e7f7b5dd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:20.968040 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-a1740b91-b784-48be-b364-349e7f7b5dd9 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:02:20.969240 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-a1740b91-b784-48be-b364-349e7f7b5dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:02:21.054005 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 264/531] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:02:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 00:02:21.479525 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdca4ee-8b4d-422d-9a6f-ad29616979eb None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:21.590592 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdca4ee-8b4d-422d-9a6f-ad29616979eb None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:21.625080 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c63f507-43c2-4ae4-9749-351cf73dfb90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:21.625706 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c63f507-43c2-4ae4-9749-351cf73dfb90 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:21.625980 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c63f507-43c2-4ae4-9749-351cf73dfb90 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:21.626245 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c63f507-43c2-4ae4-9749-351cf73dfb90 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:21.636649 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c63f507-43c2-4ae4-9749-351cf73dfb90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 00:02:21.641413 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-0c63f507-43c2-4ae4-9749-351cf73dfb90 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 00:02:21.641413 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:21.641413 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:21.641413 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:21.641413 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:21.641413 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:21.641413 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:21.641413 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:21.641413 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:21.641413 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:21.641413 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:21.641413 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:21.641413 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 00:02:21.641413 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 00:02:21.641413 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 00:02:21.641413 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 00:02:21.642326 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 00:02:21.642326 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 00:02:21.642326 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 00:02:21.642326 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 00:02:21.642326 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:21.642326 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:21.642326 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:21.642326 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:21.642326 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:21.642326 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:21.642326 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:21.642326 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:21.642326 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:21.642326 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:21.642326 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:21.642326 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:21.644100 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:21.644100 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:21.644100 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:21.644100 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:21.644100 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 00:02:21.644100 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 00:02:21.644100 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 00:02:21.644100 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 00:02:21.644100 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 00:02:21.644100 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 00:02:21.644100 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 00:02:21.644100 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:21.644100 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 268/532] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Oct 12 00:02:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:21.650816 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1740b91-b784-48be-b364-349e7f7b5dd9 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:21.691788 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:21.753629 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1740b91-b784-48be-b364-349e7f7b5dd9 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:21.790559 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9f1b2c-a232-435b-afa9-b3423f788f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:21.791171 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9f1b2c-a232-435b-afa9-b3423f788f4e None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:21.791467 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9f1b2c-a232-435b-afa9-b3423f788f4e None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:21.791737 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9f1b2c-a232-435b-afa9-b3423f788f4e None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:21.792395 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9f1b2c-a232-435b-afa9-b3423f788f4e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:21.792691 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9f1b2c-a232-435b-afa9-b3423f788f4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:21.794053 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9f1b2c-a232-435b-afa9-b3423f788f4e None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:21.802868 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 265/533] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Oct 12 00:02:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:21.808434 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c63f507-43c2-4ae4-9749-351cf73dfb90 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:21.819862 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:21.933929 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c63f507-43c2-4ae4-9749-351cf73dfb90 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:21.967429 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b454f7-876e-4353-b99b-573b3ce2b7cf None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:21.968063 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b454f7-876e-4353-b99b-573b3ce2b7cf None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:21.968063 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b454f7-876e-4353-b99b-573b3ce2b7cf None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:21.969310 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 266/534] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:02:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 00:02:21.971942 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d88d15-2c21-4178-8782-967cb8794ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:21.972565 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d88d15-2c21-4178-8782-967cb8794ffa None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:21.972836 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d88d15-2c21-4178-8782-967cb8794ffa None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:21.973102 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d88d15-2c21-4178-8782-967cb8794ffa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:21.981382 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2a8195-09a1-4c34-931a-738d6aff5665 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:21.981673 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2a8195-09a1-4c34-931a-738d6aff5665 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:21.981944 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2a8195-09a1-4c34-931a-738d6aff5665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:21.983330 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d88d15-2c21-4178-8782-967cb8794ffa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:21.984886 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d88d15-2c21-4178-8782-967cb8794ffa None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:22.000927 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-c7d88d15-2c21-4178-8782-967cb8794ffa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:22.000927 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:22.000927 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:22.000927 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:22.000927 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:22.000927 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:22.000927 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:22.000927 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:22.000927 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:22.000927 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:22.000927 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:22.000927 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:22.000927 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 00:02:22.000927 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 00:02:22.000927 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 00:02:22.000927 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 00:02:22.003482 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:22.003482 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:22.003482 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:22.003482 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:22.003482 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:22.003482 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:22.003482 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:22.003482 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:22.003482 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:22.003482 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:22.003482 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:22.003482 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:22.003482 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:22.003482 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:22.003482 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:22.003482 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:22.003482 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 00:02:22.004701 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 00:02:22.004701 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:22.004701 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:22.004701 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 269/535] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:02:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:22.011894 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d88d15-2c21-4178-8782-967cb8794ffa None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:22.024866 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-ce2a8195-09a1-4c34-931a-738d6aff5665 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:02:22.026239 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-ce2a8195-09a1-4c34-931a-738d6aff5665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:02:22.115400 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:22.148970 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 267/536] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:02:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 00:02:22.182334 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d88d15-2c21-4178-8782-967cb8794ffa None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:22.229236 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc5d5d3-0cbf-47ee-aa97-db77e6058516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:22.229853 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc5d5d3-0cbf-47ee-aa97-db77e6058516 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:22.230132 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc5d5d3-0cbf-47ee-aa97-db77e6058516 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:22.230400 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc5d5d3-0cbf-47ee-aa97-db77e6058516 None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:22.231111 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc5d5d3-0cbf-47ee-aa97-db77e6058516 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:22.231436 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc5d5d3-0cbf-47ee-aa97-db77e6058516 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:22.232814 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc5d5d3-0cbf-47ee-aa97-db77e6058516 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:22.249429 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 270/537] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Oct 12 00:02:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:22.255303 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2a8195-09a1-4c34-931a-738d6aff5665 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:22.293752 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:22.388232 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2a8195-09a1-4c34-931a-738d6aff5665 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:22.437966 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f770ced9-98fa-40e1-87ff-d7bfc22dd99c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:22.438858 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f770ced9-98fa-40e1-87ff-d7bfc22dd99c None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:22.439285 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f770ced9-98fa-40e1-87ff-d7bfc22dd99c None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:22.439756 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f770ced9-98fa-40e1-87ff-d7bfc22dd99c None admin] PATH_INFO: `/v3/users/swift` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:22.453001 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f770ced9-98fa-40e1-87ff-d7bfc22dd99c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:22.454653 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f770ced9-98fa-40e1-87ff-d7bfc22dd99c None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:22.469487 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-f770ced9-98fa-40e1-87ff-d7bfc22dd99c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 12 00:02:22.469487 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:22.469487 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:22.469487 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:22.469487 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:22.469487 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:22.469487 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:22.469487 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:22.469487 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:22.469487 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:22.469487 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:22.469487 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:22.469487 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 00:02:22.469487 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 00:02:22.469487 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 00:02:22.469487 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 00:02:22.470635 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:22.470635 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:22.470635 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 00:02:22.470635 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 00:02:22.470635 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 00:02:22.470635 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 00:02:22.470635 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:22.470635 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:22.470635 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:22.470635 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:22.470635 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:22.470635 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:22.470635 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:22.470635 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:22.470635 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:22.470635 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:22.470635 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:22.470635 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:22.472466 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:22.472466 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:22.472466 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 00:02:22.472466 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 00:02:22.472466 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 00:02:22.472466 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 00:02:22.472466 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 00:02:22.472466 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 00:02:22.472466 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 12 00:02:22.472466 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:22.474580 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 268/538] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Oct 12 00:02:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:22.480283 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc5d5d3-0cbf-47ee-aa97-db77e6058516 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:22.617641 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc5d5d3-0cbf-47ee-aa97-db77e6058516 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:22.619182 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f770ced9-98fa-40e1-87ff-d7bfc22dd99c None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:22.635711 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:22.661174 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbf1a66-957f-4c9e-9ac8-bc069441b1b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:22.661786 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf1a66-957f-4c9e-9ac8-bc069441b1b9 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:22.667516 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf1a66-957f-4c9e-9ac8-bc069441b1b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:22.667516 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf1a66-957f-4c9e-9ac8-bc069441b1b9 None admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:22.667516 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbf1a66-957f-4c9e-9ac8-bc069441b1b9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:22.667516 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbf1a66-957f-4c9e-9ac8-bc069441b1b9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:22.667516 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbf1a66-957f-4c9e-9ac8-bc069441b1b9 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:22.707477 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 271/539] 10.222.0.36 () {56 vars in 1135 bytes} [Wed Oct 12 00:02:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:22.713405 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbf1a66-957f-4c9e-9ac8-bc069441b1b9 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:22.798931 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:22.809899 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f770ced9-98fa-40e1-87ff-d7bfc22dd99c None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:22.850040 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132f347f-5c34-405a-b7b3-d54ec1255574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:22.850697 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f347f-5c34-405a-b7b3-d54ec1255574 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:22.850975 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f347f-5c34-405a-b7b3-d54ec1255574 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:22.851243 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f347f-5c34-405a-b7b3-d54ec1255574 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:22.863718 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132f347f-5c34-405a-b7b3-d54ec1255574 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 00:02:22.868830 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-132f347f-5c34-405a-b7b3-d54ec1255574 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 00:02:22.868830 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:22.868830 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:22.868830 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:22.868830 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:22.868830 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:22.868830 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:22.868830 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:22.868830 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:22.868830 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:22.868830 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:22.868830 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:22.868830 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 00:02:22.868830 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 00:02:22.868830 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 00:02:22.868830 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 00:02:22.869993 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 00:02:22.869993 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 00:02:22.869993 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 00:02:22.869993 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 00:02:22.869993 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:22.869993 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:22.869993 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:22.869993 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:22.869993 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:22.869993 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:22.869993 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:22.869993 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:22.869993 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:22.869993 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:22.869993 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:22.869993 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:22.870891 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:22.870891 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:22.870891 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:22.870891 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:22.870891 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 00:02:22.870891 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 00:02:22.870891 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 00:02:22.870891 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 00:02:22.870891 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 00:02:22.870891 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 00:02:22.870891 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 00:02:22.870891 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:22.870891 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 269/540] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Oct 12 00:02:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:22.871601 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589b10d5-d9db-44d1-9951-6acf3af359ee None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:22.871843 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589b10d5-d9db-44d1-9951-6acf3af359ee None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:22.872112 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589b10d5-d9db-44d1-9951-6acf3af359ee None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:22.878085 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 270/541] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:02:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:02:22.878085 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589b10d5-d9db-44d1-9951-6acf3af359ee None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:22.891426 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbf1a66-957f-4c9e-9ac8-bc069441b1b9 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:22.929050 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e89434-97ba-4d3c-83f2-f9109494cd13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:22.929683 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e89434-97ba-4d3c-83f2-f9109494cd13 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:22.929966 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e89434-97ba-4d3c-83f2-f9109494cd13 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:22.930229 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e89434-97ba-4d3c-83f2-f9109494cd13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:22.941205 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e89434-97ba-4d3c-83f2-f9109494cd13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:22.942645 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e89434-97ba-4d3c-83f2-f9109494cd13 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:22.958313 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-f0e89434-97ba-4d3c-83f2-f9109494cd13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:22.958313 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:22.958313 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:22.958313 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:22.958313 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:22.958313 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:22.958313 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:22.958313 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:22.958313 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:22.958313 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:22.958313 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:22.958313 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:22.958313 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 00:02:22.958313 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 00:02:22.958313 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 00:02:22.958313 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 00:02:22.959318 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:22.959318 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:22.959318 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:22.959318 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:22.959318 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:22.959318 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:22.959318 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:22.959318 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:22.959318 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:22.959318 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:22.959318 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:22.959318 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:22.959318 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:22.959318 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:22.959318 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:22.959318 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:22.959318 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 00:02:22.960374 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 00:02:22.960374 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:22.960374 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:22.960374 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 272/542] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:02:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 00:02:22.961576 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2344292a-a7c9-4d01-a41f-cb07f4e84a10 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:22.961851 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2344292a-a7c9-4d01-a41f-cb07f4e84a10 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:22.962120 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2344292a-a7c9-4d01-a41f-cb07f4e84a10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:22.979430 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-2344292a-a7c9-4d01-a41f-cb07f4e84a10 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:02:22.980624 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-2344292a-a7c9-4d01-a41f-cb07f4e84a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:02:22.992763 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589b10d5-d9db-44d1-9951-6acf3af359ee None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:23.003395 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:23.027208 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec59b48-e21c-4f44-8ae9-2b4e07ca4d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:23.027839 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec59b48-e21c-4f44-8ae9-2b4e07ca4d91 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:23.028166 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec59b48-e21c-4f44-8ae9-2b4e07ca4d91 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:23.028452 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec59b48-e21c-4f44-8ae9-2b4e07ca4d91 None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:23.029111 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec59b48-e21c-4f44-8ae9-2b4e07ca4d91 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:23.029411 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec59b48-e21c-4f44-8ae9-2b4e07ca4d91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:23.030772 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec59b48-e21c-4f44-8ae9-2b4e07ca4d91 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:23.038995 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 271/543] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Oct 12 00:02:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:23.040180 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5340f681-b7cc-4de6-a21a-c0fa16b12910 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:23.040461 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5340f681-b7cc-4de6-a21a-c0fa16b12910 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:23.040738 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5340f681-b7cc-4de6-a21a-c0fa16b12910 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:23.041913 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 272/544] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:02:23 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:02:23.042913 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7b046d-b898-491a-8d4f-cb375067f9c6 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:23.043190 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7b046d-b898-491a-8d4f-cb375067f9c6 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:23.043483 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7b046d-b898-491a-8d4f-cb375067f9c6 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:23.044889 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 273/545] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:02:23 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:02:23.045595 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7b046d-b898-491a-8d4f-cb375067f9c6 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:23.053011 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 273/546] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:02:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 00:02:23.053786 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2344292a-a7c9-4d01-a41f-cb07f4e84a10 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:23.060292 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:23.157698 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7b046d-b898-491a-8d4f-cb375067f9c6 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:23.163099 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2344292a-a7c9-4d01-a41f-cb07f4e84a10 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:23.194208 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469746b9-a368-42e0-934a-258b440509d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:23.194816 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469746b9-a368-42e0-934a-258b440509d0 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:23.195230 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469746b9-a368-42e0-934a-258b440509d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:23.195422 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469746b9-a368-42e0-934a-258b440509d0 None admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:23.196021 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469746b9-a368-42e0-934a-258b440509d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:23.196311 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469746b9-a368-42e0-934a-258b440509d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:23.197732 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469746b9-a368-42e0-934a-258b440509d0 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:23.202344 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98278c4-2a08-4ef2-a446-932e86337a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:23.202970 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98278c4-2a08-4ef2-a446-932e86337a06 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:23.203249 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98278c4-2a08-4ef2-a446-932e86337a06 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:23.203547 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98278c4-2a08-4ef2-a446-932e86337a06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:23.214510 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 274/547] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Oct 12 00:02:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:23.214510 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b30710-df32-4f5a-ae25-73a172d116ed None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:23.214510 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b30710-df32-4f5a-ae25-73a172d116ed None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:23.214510 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b30710-df32-4f5a-ae25-73a172d116ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:23.219283 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98278c4-2a08-4ef2-a446-932e86337a06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:23.220786 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98278c4-2a08-4ef2-a446-932e86337a06 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:23.231057 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-f2b30710-df32-4f5a-ae25-73a172d116ed None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:02:23.232277 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-f2b30710-df32-4f5a-ae25-73a172d116ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:02:23.237108 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-c98278c4-2a08-4ef2-a446-932e86337a06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:23.237108 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:23.237108 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:23.237108 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:23.237108 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:23.237108 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:23.237108 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:23.237108 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:23.237108 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:23.237108 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:23.237108 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:23.237108 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:23.237108 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 00:02:23.237108 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 00:02:23.237108 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 00:02:23.237108 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 00:02:23.238096 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:23.238096 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:23.238096 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:23.238096 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:23.238096 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:23.238096 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:23.238096 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:23.238096 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:23.238096 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:23.238096 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:23.238096 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:23.238096 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:23.238096 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:23.238096 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:23.238096 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:23.238096 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:23.238096 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 00:02:23.239047 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 00:02:23.239047 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:23.239047 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:23.239047 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 274/548] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:02:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:23.239937 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deafcaa-a70e-4944-a9f0-43d5d444c5ff None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:23.240212 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deafcaa-a70e-4944-a9f0-43d5d444c5ff None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:23.240481 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deafcaa-a70e-4944-a9f0-43d5d444c5ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:23.258486 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-1deafcaa-a70e-4944-a9f0-43d5d444c5ff None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:02:23.259953 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-1deafcaa-a70e-4944-a9f0-43d5d444c5ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:02:23.262776 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:23.315505 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 275/549] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:02:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 00:02:23.315505 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b30710-df32-4f5a-ae25-73a172d116ed None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:23.316254 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:23.343765 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 275/550] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:02:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 00:02:23.344792 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deafcaa-a70e-4944-a9f0-43d5d444c5ff None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:23.431726 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b30710-df32-4f5a-ae25-73a172d116ed None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:23.459065 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deafcaa-a70e-4944-a9f0-43d5d444c5ff None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:23.475672 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f33d460-1e6a-45a8-958c-c47b91c83848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:23.476837 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f33d460-1e6a-45a8-958c-c47b91c83848 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:23.477186 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f33d460-1e6a-45a8-958c-c47b91c83848 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:23.477638 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f33d460-1e6a-45a8-958c-c47b91c83848 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:23.486034 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f33d460-1e6a-45a8-958c-c47b91c83848 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:23.488271 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f33d460-1e6a-45a8-958c-c47b91c83848 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:23.498057 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9498acb-7b2a-4a0a-8d13-d06b42ca1fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:23.498834 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9498acb-7b2a-4a0a-8d13-d06b42ca1fea None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:23.499291 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9498acb-7b2a-4a0a-8d13-d06b42ca1fea None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:23.499728 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9498acb-7b2a-4a0a-8d13-d06b42ca1fea None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:23.500529 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9498acb-7b2a-4a0a-8d13-d06b42ca1fea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:23.501226 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9498acb-7b2a-4a0a-8d13-d06b42ca1fea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:23.502863 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-9f33d460-1e6a-45a8-958c-c47b91c83848 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 00:02:23.502863 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:23.502863 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:23.502863 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:23.502863 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:23.502863 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:23.502863 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:23.502863 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:23.502863 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:23.502863 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:23.502863 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:23.502863 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:23.502863 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 00:02:23.502863 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 00:02:23.502863 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 00:02:23.503954 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 00:02:23.503954 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:23.503954 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:23.503954 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:23.503954 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:23.503954 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:23.503954 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:23.503954 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:23.503954 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:23.503954 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:23.503954 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:23.503954 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:23.503954 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:23.503954 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:23.503954 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:23.503954 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:23.503954 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:23.505060 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 00:02:23.505060 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 00:02:23.505060 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 00:02:23.505060 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 00:02:23.505060 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 00:02:23.505060 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 00:02:23.505060 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 00:02:23.505060 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:23.512657 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 276/551] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Oct 12 00:02:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:23.512657 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f33d460-1e6a-45a8-958c-c47b91c83848 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:23.513634 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9498acb-7b2a-4a0a-8d13-d06b42ca1fea None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:23.523940 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:23.528389 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 276/552] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Oct 12 00:02:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:23.529471 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9498acb-7b2a-4a0a-8d13-d06b42ca1fea None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:23.597040 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:23.636498 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f33d460-1e6a-45a8-958c-c47b91c83848 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:23.648512 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9498acb-7b2a-4a0a-8d13-d06b42ca1fea None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:23.669963 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fa7f5f-2f37-4c52-8995-262923d015a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:23.670569 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fa7f5f-2f37-4c52-8995-262923d015a5 None admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:23.670848 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fa7f5f-2f37-4c52-8995-262923d015a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:23.671117 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fa7f5f-2f37-4c52-8995-262923d015a5 None admin] PATH_INFO: `/v3/services` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:23.671793 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fa7f5f-2f37-4c52-8995-262923d015a5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:23.673163 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fa7f5f-2f37-4c52-8995-262923d015a5 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:23.680865 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271a7f27-bd91-4836-98b4-7f7876964e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:23.681464 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271a7f27-bd91-4836-98b4-7f7876964e62 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:23.681740 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271a7f27-bd91-4836-98b4-7f7876964e62 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:23.682010 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271a7f27-bd91-4836-98b4-7f7876964e62 None admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:23.682692 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271a7f27-bd91-4836-98b4-7f7876964e62 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:23.682995 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271a7f27-bd91-4836-98b4-7f7876964e62 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:23.684461 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271a7f27-bd91-4836-98b4-7f7876964e62 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:23.696642 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 277/553] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Oct 12 00:02:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:23.697439 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271a7f27-bd91-4836-98b4-7f7876964e62 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:23.756459 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:23.779487 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 277/554] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Oct 12 00:02:23 2022] POST /identity/v3/services => generated 251 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:02:23.779487 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fa7f5f-2f37-4c52-8995-262923d015a5 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:23.803147 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271a7f27-bd91-4836-98b4-7f7876964e62 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:23.858382 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8a1807-20af-4126-8e23-0fa459b60fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:23.858946 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a1807-20af-4126-8e23-0fa459b60fb9 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:23.859270 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a1807-20af-4126-8e23-0fa459b60fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:23.859566 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a1807-20af-4126-8e23-0fa459b60fb9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:23.868665 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8a1807-20af-4126-8e23-0fa459b60fb9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:23.870234 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8a1807-20af-4126-8e23-0fa459b60fb9 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:23.884813 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-bf8a1807-20af-4126-8e23-0fa459b60fb9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 12 00:02:23.884813 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:23.884813 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:23.884813 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:23.884813 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:23.884813 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:23.884813 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:23.884813 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:23.884813 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:23.884813 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:23.884813 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:23.884813 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:23.884813 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 00:02:23.884813 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 00:02:23.884813 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 00:02:23.884813 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 00:02:23.885835 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:23.885835 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:23.885835 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 00:02:23.885835 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 00:02:23.885835 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 00:02:23.885835 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 00:02:23.885835 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:23.885835 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:23.885835 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:23.885835 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:23.885835 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:23.885835 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:23.885835 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:23.885835 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:23.885835 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:23.885835 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:23.885835 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:23.885835 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:23.886802 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:23.886802 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:23.886802 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 00:02:23.886802 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 00:02:23.886802 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 00:02:23.886802 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 00:02:23.886802 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 00:02:23.886802 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 00:02:23.886802 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 12 00:02:23.886802 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:23.886802 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 278/555] 10.222.0.36 () {56 vars in 1093 bytes} [Wed Oct 12 00:02:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:23.886802 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8a1807-20af-4126-8e23-0fa459b60fb9 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:23.899851 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fa7f5f-2f37-4c52-8995-262923d015a5 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:23.924480 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:23.939456 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf985482-0a93-4653-b8bb-0343de8ce294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:23.940639 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf985482-0a93-4653-b8bb-0343de8ce294 None admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:23.940913 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf985482-0a93-4653-b8bb-0343de8ce294 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:23.941177 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf985482-0a93-4653-b8bb-0343de8ce294 None admin] PATH_INFO: `/v3/services` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:23.941829 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf985482-0a93-4653-b8bb-0343de8ce294 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:23.943200 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf985482-0a93-4653-b8bb-0343de8ce294 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:23.997394 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8a1807-20af-4126-8e23-0fa459b60fb9 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:24.004432 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 278/556] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Oct 12 00:02:23 2022] POST /identity/v3/services => generated 240 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:02:24.005261 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf985482-0a93-4653-b8bb-0343de8ce294 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:24.020677 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:24.053576 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23e3495-6a19-4c62-bef6-8096d604fa83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:24.054423 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23e3495-6a19-4c62-bef6-8096d604fa83 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:24.055332 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23e3495-6a19-4c62-bef6-8096d604fa83 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:24.055860 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23e3495-6a19-4c62-bef6-8096d604fa83 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:24.072634 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23e3495-6a19-4c62-bef6-8096d604fa83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 00:02:24.079690 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-d23e3495-6a19-4c62-bef6-8096d604fa83 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 12 00:02:24.079690 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:24.079690 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:24.079690 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:24.079690 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:24.079690 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:24.079690 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:24.079690 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:24.079690 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:24.079690 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:24.079690 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:24.079690 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:24.079690 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 00:02:24.079690 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 00:02:24.079690 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 00:02:24.079690 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 00:02:24.080804 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 00:02:24.080804 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 00:02:24.080804 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 00:02:24.080804 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 00:02:24.080804 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:24.080804 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:24.080804 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:24.080804 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:24.080804 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:24.080804 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:24.080804 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:24.080804 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:24.080804 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:24.080804 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:24.080804 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:24.080804 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:24.081621 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:24.081621 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:24.081621 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:24.081621 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:24.081621 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 00:02:24.081621 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 00:02:24.081621 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 00:02:24.081621 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 00:02:24.081621 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 00:02:24.081621 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 00:02:24.081621 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 12 00:02:24.081621 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:24.086056 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 279/557] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Oct 12 00:02:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:24.087790 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23e3495-6a19-4c62-bef6-8096d604fa83 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:24.154614 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf985482-0a93-4653-b8bb-0343de8ce294 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:24.198322 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1239d35-1d37-4073-b5d9-c3b6539ac844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:24.198993 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1239d35-1d37-4073-b5d9-c3b6539ac844 None admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:24.199276 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1239d35-1d37-4073-b5d9-c3b6539ac844 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:24.199571 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1239d35-1d37-4073-b5d9-c3b6539ac844 None admin] PATH_INFO: `/v3/projects/0697927c6aac43d4b14a750fbbb1145e/users/fc1bcffc78844dcbb1a9b7c80285249b/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:24.201670 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1239d35-1d37-4073-b5d9-c3b6539ac844 None admin] RBAC: Authorizing `identity:create_grant(project_id=0697927c6aac43d4b14a750fbbb1145e, user_id=fc1bcffc78844dcbb1a9b7c80285249b, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:24.203104 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1239d35-1d37-4073-b5d9-c3b6539ac844 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:24.223815 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23e3495-6a19-4c62-bef6-8096d604fa83 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:24.261844 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdd3076-1fcf-46ba-95a6-c9defbfa982f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:24.262456 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdd3076-1fcf-46ba-95a6-c9defbfa982f None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:24.262723 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdd3076-1fcf-46ba-95a6-c9defbfa982f None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:24.262996 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdd3076-1fcf-46ba-95a6-c9defbfa982f None admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:24.263863 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdd3076-1fcf-46ba-95a6-c9defbfa982f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:24.264165 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdd3076-1fcf-46ba-95a6-c9defbfa982f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:24.265603 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdd3076-1fcf-46ba-95a6-c9defbfa982f None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:24.286685 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 279/558] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Oct 12 00:02:24 2022] PUT /identity/v3/projects/0697927c6aac43d4b14a750fbbb1145e/users/fc1bcffc78844dcbb1a9b7c80285249b/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:02:24.287855 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1239d35-1d37-4073-b5d9-c3b6539ac844 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:24.291390 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 280/559] 10.222.0.36 () {56 vars in 1133 bytes} [Wed Oct 12 00:02:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:24.294184 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdd3076-1fcf-46ba-95a6-c9defbfa982f None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:24.363597 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:24.384534 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:24.499661 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1239d35-1d37-4073-b5d9-c3b6539ac844 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:24.509646 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdd3076-1fcf-46ba-95a6-c9defbfa982f None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:24.547950 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e54357-7e7d-4ede-a035-2a7141c7aee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:24.548751 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e54357-7e7d-4ede-a035-2a7141c7aee6 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:24.549211 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e54357-7e7d-4ede-a035-2a7141c7aee6 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:24.549614 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e54357-7e7d-4ede-a035-2a7141c7aee6 None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:24.550529 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e54357-7e7d-4ede-a035-2a7141c7aee6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:24.550877 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3995ef5e-2b50-48b6-876c-3d407d2cc5ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:24.551123 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e54357-7e7d-4ede-a035-2a7141c7aee6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:24.551548 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3995ef5e-2b50-48b6-876c-3d407d2cc5ba None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:24.551830 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3995ef5e-2b50-48b6-876c-3d407d2cc5ba None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:24.552106 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3995ef5e-2b50-48b6-876c-3d407d2cc5ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:24.553158 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e54357-7e7d-4ede-a035-2a7141c7aee6 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:24.562789 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 280/560] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Oct 12 00:02:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:24.568858 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e54357-7e7d-4ede-a035-2a7141c7aee6 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:24.638383 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3995ef5e-2b50-48b6-876c-3d407d2cc5ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:24.640148 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3995ef5e-2b50-48b6-876c-3d407d2cc5ba None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:24.695737 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:24.782143 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e54357-7e7d-4ede-a035-2a7141c7aee6 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:24.788597 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:24.808473 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 281/561] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Oct 12 00:02:24 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 12 00:02:24.815501 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3995ef5e-2b50-48b6-876c-3d407d2cc5ba None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:24.850874 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b85769-afd8-4ca7-a199-1bb63e00d73c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:24.851642 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b85769-afd8-4ca7-a199-1bb63e00d73c None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:24.851923 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b85769-afd8-4ca7-a199-1bb63e00d73c None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:24.852194 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b85769-afd8-4ca7-a199-1bb63e00d73c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:24.874846 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b85769-afd8-4ca7-a199-1bb63e00d73c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:24.876389 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b85769-afd8-4ca7-a199-1bb63e00d73c None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:24.897060 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-64b85769-afd8-4ca7-a199-1bb63e00d73c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:24.897060 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:24.897060 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:24.897060 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:24.897060 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:24.897060 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:24.897060 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:24.897060 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:24.897060 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:24.897060 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:24.897060 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:24.897060 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:24.897060 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 00:02:24.897060 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 00:02:24.897060 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 00:02:24.897060 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 00:02:24.898035 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:24.898035 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:24.898035 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:24.898035 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:24.898035 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:24.898035 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:24.898035 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:24.898035 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:24.898035 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:24.898035 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:24.898035 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:24.898035 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:24.898035 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:24.898035 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:24.898035 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:24.898035 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:24.898035 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 00:02:24.898925 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 00:02:24.898925 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:24.898925 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:24.898925 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 281/562] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:02:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:24.904305 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b85769-afd8-4ca7-a199-1bb63e00d73c None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:24.931927 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3995ef5e-2b50-48b6-876c-3d407d2cc5ba None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:24.962911 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:24.996075 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00280573-63dd-407b-80cf-150e3865e0ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:24.996738 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00280573-63dd-407b-80cf-150e3865e0ef None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:24.999739 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00280573-63dd-407b-80cf-150e3865e0ef None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:25.000326 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00280573-63dd-407b-80cf-150e3865e0ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:25.044388 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00280573-63dd-407b-80cf-150e3865e0ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:25.045692 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00280573-63dd-407b-80cf-150e3865e0ef None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:25.050093 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b85769-afd8-4ca7-a199-1bb63e00d73c None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:25.087965 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a92f559-69bc-4405-9e0b-e0b642455ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:25.088896 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-00280573-63dd-407b-80cf-150e3865e0ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:25.088896 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:25.088896 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:25.088896 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:25.088896 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:25.088896 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:25.088896 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:25.088896 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:25.088896 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:25.088896 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:25.088896 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:25.088896 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:25.088896 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 00:02:25.088896 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 00:02:25.088896 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 00:02:25.088896 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 00:02:25.089788 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:25.089788 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:25.089788 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:25.089788 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:25.089788 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:25.089788 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:25.089788 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:25.089788 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:25.089788 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:25.089788 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:25.089788 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:25.089788 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:25.089788 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:25.089788 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:25.089788 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:25.089788 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:25.089788 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 00:02:25.090687 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 00:02:25.090687 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:25.090687 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:25.090687 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 282/563] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:02:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 00:02:25.091469 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a92f559-69bc-4405-9e0b-e0b642455ccf None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:25.091954 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a92f559-69bc-4405-9e0b-e0b642455ccf None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:25.092416 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a92f559-69bc-4405-9e0b-e0b642455ccf None admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:25.093249 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a92f559-69bc-4405-9e0b-e0b642455ccf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:25.093760 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a92f559-69bc-4405-9e0b-e0b642455ccf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:25.095469 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a92f559-69bc-4405-9e0b-e0b642455ccf None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:25.096280 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00280573-63dd-407b-80cf-150e3865e0ef None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:25.109685 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 282/564] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Oct 12 00:02:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:25.115986 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a92f559-69bc-4405-9e0b-e0b642455ccf None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:25.155776 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:25.228105 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a92f559-69bc-4405-9e0b-e0b642455ccf None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:25.266313 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e134540-c2cf-437d-a0cd-28e0130a33d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:25.267303 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e134540-c2cf-437d-a0cd-28e0130a33d7 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:25.268516 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e134540-c2cf-437d-a0cd-28e0130a33d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:25.269007 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e134540-c2cf-437d-a0cd-28e0130a33d7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:25.277907 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e134540-c2cf-437d-a0cd-28e0130a33d7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:25.279869 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e134540-c2cf-437d-a0cd-28e0130a33d7 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:25.292575 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-2e134540-c2cf-437d-a0cd-28e0130a33d7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 12 00:02:25.292575 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:25.292575 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:25.292575 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:25.292575 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:25.292575 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:25.292575 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:25.292575 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:25.292575 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:25.292575 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:25.292575 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:25.292575 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:25.292575 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 00:02:25.292575 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 00:02:25.292575 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 00:02:25.292575 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 00:02:25.293546 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:25.293546 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:25.293546 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 00:02:25.293546 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 00:02:25.293546 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 00:02:25.293546 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 00:02:25.293546 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:25.293546 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:25.293546 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:25.293546 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:25.293546 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:25.293546 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:25.293546 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:25.293546 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:25.293546 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:25.293546 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:25.293546 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:25.293546 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:25.294515 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:25.294515 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:25.294515 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 00:02:25.294515 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 00:02:25.294515 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 00:02:25.294515 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 00:02:25.294515 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 00:02:25.294515 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 00:02:25.294515 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 12 00:02:25.294515 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:25.297656 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 283/565] 10.222.0.36 () {56 vars in 1117 bytes} [Wed Oct 12 00:02:25 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:25.303059 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e134540-c2cf-437d-a0cd-28e0130a33d7 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:25.345587 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00280573-63dd-407b-80cf-150e3865e0ef None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:25.386575 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd874c60-c094-42f0-af23-02f302e3f063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:25.388067 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd874c60-c094-42f0-af23-02f302e3f063 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:25.388554 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd874c60-c094-42f0-af23-02f302e3f063 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:25.389123 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd874c60-c094-42f0-af23-02f302e3f063 None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:25.390311 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd874c60-c094-42f0-af23-02f302e3f063 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:25.391128 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd874c60-c094-42f0-af23-02f302e3f063 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:25.393305 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd874c60-c094-42f0-af23-02f302e3f063 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:25.408081 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 283/566] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Oct 12 00:02:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:25.418606 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd874c60-c094-42f0-af23-02f302e3f063 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:25.456084 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:25.496806 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:25.500824 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e134540-c2cf-437d-a0cd-28e0130a33d7 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:25.568730 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ba868e-77a4-4140-b9ca-ff45522a0f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:25.569350 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ba868e-77a4-4140-b9ca-ff45522a0f08 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:25.569622 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ba868e-77a4-4140-b9ca-ff45522a0f08 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:25.569890 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ba868e-77a4-4140-b9ca-ff45522a0f08 None admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:25.570617 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ba868e-77a4-4140-b9ca-ff45522a0f08 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:25.570917 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ba868e-77a4-4140-b9ca-ff45522a0f08 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:25.572393 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ba868e-77a4-4140-b9ca-ff45522a0f08 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:25.607999 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd874c60-c094-42f0-af23-02f302e3f063 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:25.611968 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 284/567] 10.222.0.36 () {56 vars in 1149 bytes} [Wed Oct 12 00:02:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:25.617447 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ba868e-77a4-4140-b9ca-ff45522a0f08 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:25.675012 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f5380e-32ff-4561-b092-d5f422e7e6bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:25.675867 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f5380e-32ff-4561-b092-d5f422e7e6bd None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:25.676146 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f5380e-32ff-4561-b092-d5f422e7e6bd None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:25.676414 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f5380e-32ff-4561-b092-d5f422e7e6bd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:25.691740 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f5380e-32ff-4561-b092-d5f422e7e6bd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:25.693186 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f5380e-32ff-4561-b092-d5f422e7e6bd None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:25.711596 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-16f5380e-32ff-4561-b092-d5f422e7e6bd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 00:02:25.711596 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:25.711596 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:25.711596 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:25.711596 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:25.711596 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:25.711596 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:25.711596 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:25.711596 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:25.711596 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:25.711596 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:25.711596 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:25.711596 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 00:02:25.711596 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 00:02:25.711596 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 00:02:25.712648 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 00:02:25.712648 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:25.712648 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:25.712648 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:25.712648 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:25.712648 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:25.712648 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:25.712648 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:25.712648 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:25.712648 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:25.712648 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:25.712648 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:25.712648 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:25.712648 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:25.712648 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:25.712648 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:25.712648 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:25.713641 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 00:02:25.713641 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 00:02:25.713641 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 00:02:25.713641 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 00:02:25.713641 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 00:02:25.713641 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 00:02:25.713641 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 00:02:25.713641 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:25.714891 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 284/568] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Oct 12 00:02:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:25.719918 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f5380e-32ff-4561-b092-d5f422e7e6bd None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:25.808412 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ba868e-77a4-4140-b9ca-ff45522a0f08 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:25.848059 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9abfaa8-a6be-42bb-953f-adf0b1771401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:25.848702 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9abfaa8-a6be-42bb-953f-adf0b1771401 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:25.848980 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9abfaa8-a6be-42bb-953f-adf0b1771401 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:25.849250 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9abfaa8-a6be-42bb-953f-adf0b1771401 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:25.851661 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:25.869993 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9abfaa8-a6be-42bb-953f-adf0b1771401 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:25.871501 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9abfaa8-a6be-42bb-953f-adf0b1771401 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:25.896644 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-e9abfaa8-a6be-42bb-953f-adf0b1771401 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:25.896644 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:25.896644 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:25.896644 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:25.896644 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:25.896644 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:25.896644 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:25.896644 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:25.896644 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:25.896644 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:25.896644 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:25.896644 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:25.896644 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 00:02:25.896644 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 00:02:25.896644 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 00:02:25.896644 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 00:02:25.898992 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:25.898992 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:25.898992 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:25.898992 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:25.898992 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:25.898992 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:25.898992 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:25.898992 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:25.898992 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:25.898992 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:25.898992 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:25.898992 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:25.898992 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:25.898992 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:25.898992 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:25.898992 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:25.898992 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 00:02:25.901173 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 00:02:25.901173 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:25.901173 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:25.901173 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 285/569] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:02:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 00:02:25.908865 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9abfaa8-a6be-42bb-953f-adf0b1771401 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:25.912860 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f5380e-32ff-4561-b092-d5f422e7e6bd None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:25.915648 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:25.969597 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc53191c-72c4-444e-a50c-48bd4b3e8e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:25.971246 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc53191c-72c4-444e-a50c-48bd4b3e8e3c None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:25.972472 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc53191c-72c4-444e-a50c-48bd4b3e8e3c None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:25.972889 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc53191c-72c4-444e-a50c-48bd4b3e8e3c None admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:25.977755 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc53191c-72c4-444e-a50c-48bd4b3e8e3c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:25.978228 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc53191c-72c4-444e-a50c-48bd4b3e8e3c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:25.979875 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc53191c-72c4-444e-a50c-48bd4b3e8e3c None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:26.000518 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 285/570] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Oct 12 00:02:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:26.008247 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc53191c-72c4-444e-a50c-48bd4b3e8e3c None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:26.098060 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9abfaa8-a6be-42bb-953f-adf0b1771401 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:26.171290 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc0902a-1ef8-47ac-b637-6627e48a0b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:26.175237 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc0902a-1ef8-47ac-b637-6627e48a0b01 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:26.175695 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc0902a-1ef8-47ac-b637-6627e48a0b01 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:26.176136 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc0902a-1ef8-47ac-b637-6627e48a0b01 None admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:26.176931 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc0902a-1ef8-47ac-b637-6627e48a0b01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:26.177380 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc0902a-1ef8-47ac-b637-6627e48a0b01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:26.178908 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc0902a-1ef8-47ac-b637-6627e48a0b01 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:26.201473 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 286/571] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Oct 12 00:02:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:26.204885 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc0902a-1ef8-47ac-b637-6627e48a0b01 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:26.210112 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc53191c-72c4-444e-a50c-48bd4b3e8e3c None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:26.260997 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5efbbc-fa3a-4e50-9957-774bc111ccad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:26.261701 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5efbbc-fa3a-4e50-9957-774bc111ccad None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:26.262517 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5efbbc-fa3a-4e50-9957-774bc111ccad None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:26.262910 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5efbbc-fa3a-4e50-9957-774bc111ccad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:26.263744 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5efbbc-fa3a-4e50-9957-774bc111ccad None admin] RBAC: Adding query filter params (role.id=b2e41f0d7a304dc7af8b6a232049b6a2, scope.project.id=0697927c6aac43d4b14a750fbbb1145e, user.id=a515b75aa3944912ad2364ffd7924f40) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:26.264559 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5efbbc-fa3a-4e50-9957-774bc111ccad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:26.266428 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5efbbc-fa3a-4e50-9957-774bc111ccad None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:26.268784 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:26.279463 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 286/572] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Oct 12 00:02:26 2022] GET /identity/v3/role_assignments?user.id=a515b75aa3944912ad2364ffd7924f40&scope.project.id=0697927c6aac43d4b14a750fbbb1145e&role.id=b2e41f0d7a304dc7af8b6a232049b6a2 => generated 608 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:26.287125 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fab64e-7c68-4ade-a47a-5ce987c79c66 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:26.287125 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fab64e-7c68-4ade-a47a-5ce987c79c66 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:26.287125 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fab64e-7c68-4ade-a47a-5ce987c79c66 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:26.287125 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 287/573] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:02:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:02:26.298114 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997a46ec-3fc2-4c15-953a-a0d40413317e None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:26.299449 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997a46ec-3fc2-4c15-953a-a0d40413317e None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:26.299917 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997a46ec-3fc2-4c15-953a-a0d40413317e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:26.337527 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-997a46ec-3fc2-4c15-953a-a0d40413317e None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:02:26.590417 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-997a46ec-3fc2-4c15-953a-a0d40413317e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:02:26.590417 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:26.591168 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc0902a-1ef8-47ac-b637-6627e48a0b01 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:26.591168 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebfb2f4-2fc6-45e7-b9c2-9429a1d89926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:26.591168 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebfb2f4-2fc6-45e7-b9c2-9429a1d89926 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:26.591168 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebfb2f4-2fc6-45e7-b9c2-9429a1d89926 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:26.591519 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebfb2f4-2fc6-45e7-b9c2-9429a1d89926 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:26.591519 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebfb2f4-2fc6-45e7-b9c2-9429a1d89926 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:26.591519 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebfb2f4-2fc6-45e7-b9c2-9429a1d89926 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:26.591718 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 288/574] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:02:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 12 00:02:26.591718 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dcd569-c185-4b7a-a5f3-179c47c0062e None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:26.591718 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dcd569-c185-4b7a-a5f3-179c47c0062e None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:26.591718 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dcd569-c185-4b7a-a5f3-179c47c0062e None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:26.591718 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 289/575] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:02:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:02:26.591718 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc86bfb0-e9a7-4bb9-8594-2243db4a3783 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:26.591718 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc86bfb0-e9a7-4bb9-8594-2243db4a3783 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:26.592159 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-6ebfb2f4-2fc6-45e7-b9c2-9429a1d89926 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 00:02:26.592159 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:26.592159 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:26.592159 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:26.592159 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:26.592159 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:26.592159 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:26.592159 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:26.592159 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:26.592159 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:26.592159 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:26.592159 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:26.592159 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 00:02:26.592159 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 00:02:26.592159 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 00:02:26.593030 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 00:02:26.593030 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:26.593030 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:26.593030 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:26.593030 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:26.593030 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:26.593030 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:26.593030 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:26.593030 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:26.593030 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:26.593030 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:26.593030 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:26.593030 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:26.593030 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:26.593030 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:26.593030 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:26.593030 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:26.593959 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 00:02:26.593959 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 00:02:26.593959 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 00:02:26.593959 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 00:02:26.593959 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 00:02:26.593959 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 00:02:26.593959 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 00:02:26.593959 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:26.594427 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc86bfb0-e9a7-4bb9-8594-2243db4a3783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:26.594502 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 287/576] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Oct 12 00:02:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 00:02:26.594502 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebfb2f4-2fc6-45e7-b9c2-9429a1d89926 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:26.594635 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-bc86bfb0-e9a7-4bb9-8594-2243db4a3783 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:02:26.594635 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-bc86bfb0-e9a7-4bb9-8594-2243db4a3783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:02:26.617357 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 290/577] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:02:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 12 00:02:26.644733 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebfb2f4-2fc6-45e7-b9c2-9429a1d89926 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:26.661008 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:26.682109 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e9cab0-30d4-4325-8919-7cbf60eea733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:26.682724 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e9cab0-30d4-4325-8919-7cbf60eea733 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:26.682999 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e9cab0-30d4-4325-8919-7cbf60eea733 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:26.683270 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e9cab0-30d4-4325-8919-7cbf60eea733 None admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:26.684010 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e9cab0-30d4-4325-8919-7cbf60eea733 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:26.684317 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e9cab0-30d4-4325-8919-7cbf60eea733 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:26.685894 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e9cab0-30d4-4325-8919-7cbf60eea733 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:26.699881 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 288/578] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Oct 12 00:02:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 00:02:26.705836 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc86bfb0-e9a7-4bb9-8594-2243db4a3783 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:26.768001 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e9cab0-30d4-4325-8919-7cbf60eea733 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:26.802364 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:26.832467 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc86bfb0-e9a7-4bb9-8594-2243db4a3783 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:26.868993 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05c8ec4-2c06-4008-843d-3fed833227fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:26.869863 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05c8ec4-2c06-4008-843d-3fed833227fc None admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:26.870576 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05c8ec4-2c06-4008-843d-3fed833227fc None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:26.870914 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05c8ec4-2c06-4008-843d-3fed833227fc None admin] PATH_INFO: `/v3/projects/0697927c6aac43d4b14a750fbbb1145e/users/d030d18b595144f19d6ad01d4cd5d52b/roles/a53c0c4b8b8645b6a1cfe96266daec8e` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:26.888789 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05c8ec4-2c06-4008-843d-3fed833227fc None admin] RBAC: Authorizing `identity:create_grant(project_id=0697927c6aac43d4b14a750fbbb1145e, user_id=d030d18b595144f19d6ad01d4cd5d52b, role_id=a53c0c4b8b8645b6a1cfe96266daec8e)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:26.890352 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05c8ec4-2c06-4008-843d-3fed833227fc None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:26.913550 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e9cab0-30d4-4325-8919-7cbf60eea733 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:26.949142 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858b44a3-6dd1-4c5d-86b5-681dbb103dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:26.949767 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b44a3-6dd1-4c5d-86b5-681dbb103dc4 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:26.950060 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b44a3-6dd1-4c5d-86b5-681dbb103dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:26.950328 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b44a3-6dd1-4c5d-86b5-681dbb103dc4 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:26.950898 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858b44a3-6dd1-4c5d-86b5-681dbb103dc4 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:26.952300 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858b44a3-6dd1-4c5d-86b5-681dbb103dc4 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:26.961522 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-858b44a3-6dd1-4c5d-86b5-681dbb103dc4 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 12 00:02:26.961522 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:26.961522 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 00:02:26.961522 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 00:02:26.961522 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 00:02:26.961522 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 00:02:26.961522 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 00:02:26.961522 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 00:02:26.961522 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 12 00:02:26.961522 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:26.961522 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 00:02:26.961522 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:26.961522 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:26.961522 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:26.961522 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:26.961522 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:26.961522 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:26.965147 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:26.965147 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:26.965147 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:26.965147 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:26.965147 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:26.965147 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:26.965147 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 00:02:26.965147 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 00:02:26.965147 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 00:02:26.965147 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 00:02:26.965147 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:26.965147 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:26.965147 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:26.965147 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:26.965147 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:26.965147 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:26.966355 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:26.966355 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:26.966355 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:26.966355 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:26.966355 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:26.966355 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:26.966355 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:26.966355 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:26.966355 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:26.966355 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:26.966355 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 00:02:26.966355 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 00:02:26.966355 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 12 00:02:26.966355 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:26.966355 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 289/579] 10.222.0.36 () {56 vars in 1129 bytes} [Wed Oct 12 00:02:26 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:26.967671 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9df3e3-958e-49fc-a45f-b8f875deb8d7 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:26.967671 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9df3e3-958e-49fc-a45f-b8f875deb8d7 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:26.967671 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9df3e3-958e-49fc-a45f-b8f875deb8d7 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:26.967671 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 290/580] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:02:26 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:02:26.972041 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582814ac-5fb5-4c20-ba77-768bfc0b6ed7 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:26.972314 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582814ac-5fb5-4c20-ba77-768bfc0b6ed7 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:26.972582 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582814ac-5fb5-4c20-ba77-768bfc0b6ed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:26.989377 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 291/581] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Oct 12 00:02:26 2022] PUT /identity/v3/projects/0697927c6aac43d4b14a750fbbb1145e/users/d030d18b595144f19d6ad01d4cd5d52b/roles/a53c0c4b8b8645b6a1cfe96266daec8e => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 00:02:26.990729 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05c8ec4-2c06-4008-843d-3fed833227fc None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:27.001648 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-582814ac-5fb5-4c20-ba77-768bfc0b6ed7 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:02:27.003437 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-582814ac-5fb5-4c20-ba77-768bfc0b6ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:02:27.032106 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:27.088398 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 291/582] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:02:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 00:02:27.089578 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582814ac-5fb5-4c20-ba77-768bfc0b6ed7 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:27.120437 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05c8ec4-2c06-4008-843d-3fed833227fc None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:27.174450 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd541717-4b96-4bb9-a9d1-eb6ea7d4fe49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:27.175092 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd541717-4b96-4bb9-a9d1-eb6ea7d4fe49 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:27.175390 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd541717-4b96-4bb9-a9d1-eb6ea7d4fe49 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:27.175664 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd541717-4b96-4bb9-a9d1-eb6ea7d4fe49 None admin] PATH_INFO: `/v3/services` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:27.176376 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd541717-4b96-4bb9-a9d1-eb6ea7d4fe49 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:27.176751 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd541717-4b96-4bb9-a9d1-eb6ea7d4fe49 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:27.178321 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd541717-4b96-4bb9-a9d1-eb6ea7d4fe49 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:27.184738 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:27.188399 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 292/583] 10.222.0.36 () {56 vars in 1124 bytes} [Wed Oct 12 00:02:26 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:27.194859 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd541717-4b96-4bb9-a9d1-eb6ea7d4fe49 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:27.203993 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582814ac-5fb5-4c20-ba77-768bfc0b6ed7 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:27.241355 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918f799a-7e75-48e1-849e-c19bd69aec0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:27.242359 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f799a-7e75-48e1-849e-c19bd69aec0c None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:27.242969 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f799a-7e75-48e1-849e-c19bd69aec0c None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:27.243460 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f799a-7e75-48e1-849e-c19bd69aec0c None admin] PATH_INFO: `/v3/services/network` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:27.244225 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918f799a-7e75-48e1-849e-c19bd69aec0c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:27.245856 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918f799a-7e75-48e1-849e-c19bd69aec0c None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:27.255568 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-918f799a-7e75-48e1-849e-c19bd69aec0c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 12 00:02:27.255568 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:27.255568 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 00:02:27.255568 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 00:02:27.255568 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 00:02:27.255568 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 00:02:27.255568 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 00:02:27.255568 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 00:02:27.255568 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 12 00:02:27.255568 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:27.255568 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 00:02:27.255568 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:27.255568 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:27.255568 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:27.255568 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:27.255568 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:27.255568 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:27.257093 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:27.257093 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:27.257093 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:27.257093 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:27.257093 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:27.257093 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:27.257093 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 00:02:27.257093 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 00:02:27.257093 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 00:02:27.257093 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 00:02:27.257093 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:27.257093 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:27.257093 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:27.257093 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:27.257093 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:27.257093 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:27.258293 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:27.258293 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:27.258293 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:27.258293 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:27.258293 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:27.258293 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:27.258293 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:27.258293 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:27.258293 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:27.258293 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:27.258293 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 00:02:27.258293 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 00:02:27.258293 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 12 00:02:27.258293 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:27.258293 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 292/584] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Oct 12 00:02:27 2022] GET /identity/v3/services/network => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:27.263063 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918f799a-7e75-48e1-849e-c19bd69aec0c None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:27.294410 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:27.304361 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd541717-4b96-4bb9-a9d1-eb6ea7d4fe49 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:27.344130 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3942071b-6fdb-49fe-be83-6d181116e405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:27.344749 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3942071b-6fdb-49fe-be83-6d181116e405 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:27.345026 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3942071b-6fdb-49fe-be83-6d181116e405 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:27.345295 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3942071b-6fdb-49fe-be83-6d181116e405 None admin] PATH_INFO: `/v3/services` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:27.345966 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3942071b-6fdb-49fe-be83-6d181116e405 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:27.346265 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3942071b-6fdb-49fe-be83-6d181116e405 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:27.347674 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3942071b-6fdb-49fe-be83-6d181116e405 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:27.366662 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 293/585] 10.222.0.36 () {56 vars in 1124 bytes} [Wed Oct 12 00:02:27 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 00:02:27.372720 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3942071b-6fdb-49fe-be83-6d181116e405 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:27.390317 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918f799a-7e75-48e1-849e-c19bd69aec0c None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:27.450810 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9e7ebe-fb74-4722-acef-56d16f89e73a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:27.451444 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9e7ebe-fb74-4722-acef-56d16f89e73a None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:27.451723 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9e7ebe-fb74-4722-acef-56d16f89e73a None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:27.451997 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9e7ebe-fb74-4722-acef-56d16f89e73a None admin] PATH_INFO: `/v3/services` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:27.452632 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9e7ebe-fb74-4722-acef-56d16f89e73a None admin] RBAC: Adding query filter params (name=network) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:27.452931 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9e7ebe-fb74-4722-acef-56d16f89e73a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:27.454376 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9e7ebe-fb74-4722-acef-56d16f89e73a None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:27.459996 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 293/586] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Oct 12 00:02:27 2022] GET /identity/v3/services?name=network => generated 124 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:27.465255 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9e7ebe-fb74-4722-acef-56d16f89e73a None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:27.466675 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:27.509383 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3942071b-6fdb-49fe-be83-6d181116e405 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:27.561218 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670c7d9e-e484-4a73-94de-82a2485c00d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:27.562206 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670c7d9e-e484-4a73-94de-82a2485c00d1 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:27.562678 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670c7d9e-e484-4a73-94de-82a2485c00d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:27.563152 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670c7d9e-e484-4a73-94de-82a2485c00d1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:27.564116 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670c7d9e-e484-4a73-94de-82a2485c00d1 None admin] RBAC: Adding query filter params (interface=public, service_id=62da91ee53f84ae29d1300ddcd8f7c94, region_id=RegionOne) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:27.565175 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670c7d9e-e484-4a73-94de-82a2485c00d1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:27.566818 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670c7d9e-e484-4a73-94de-82a2485c00d1 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:27.571641 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:27.576189 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 294/587] 10.222.0.36 () {56 vars in 1251 bytes} [Wed Oct 12 00:02:27 2022] GET /identity/v3/endpoints?service_id=62da91ee53f84ae29d1300ddcd8f7c94&interface=public®ion_id=RegionOne => generated 194 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 00:02:27.577354 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670c7d9e-e484-4a73-94de-82a2485c00d1 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:27.594398 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9e7ebe-fb74-4722-acef-56d16f89e73a None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:27.641263 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f47ef6-d403-468e-be17-6ed3bc6106bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:27.641822 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f47ef6-d403-468e-be17-6ed3bc6106bc None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:27.642104 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f47ef6-d403-468e-be17-6ed3bc6106bc None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:27.642893 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f47ef6-d403-468e-be17-6ed3bc6106bc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:27.656277 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f47ef6-d403-468e-be17-6ed3bc6106bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 00:02:27.663501 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-30f47ef6-d403-468e-be17-6ed3bc6106bc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 00:02:27.663501 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:27.663501 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:27.663501 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:27.663501 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:27.663501 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:27.663501 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:27.663501 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:27.663501 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:27.663501 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:27.663501 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:27.663501 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:27.663501 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 00:02:27.663501 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 00:02:27.663501 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 00:02:27.663501 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 00:02:27.664456 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 00:02:27.664456 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 00:02:27.664456 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 00:02:27.664456 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 00:02:27.664456 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:27.664456 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:27.664456 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:27.664456 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:27.664456 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:27.664456 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:27.664456 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:27.664456 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:27.664456 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:27.664456 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:27.664456 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:27.664456 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:27.665324 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:27.665324 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:27.665324 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:27.665324 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:27.665324 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 00:02:27.665324 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 00:02:27.665324 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 00:02:27.665324 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 00:02:27.665324 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 00:02:27.665324 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 00:02:27.665324 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 00:02:27.665324 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:27.667507 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 294/588] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Oct 12 00:02:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:27.668538 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f47ef6-d403-468e-be17-6ed3bc6106bc None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:27.702761 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670c7d9e-e484-4a73-94de-82a2485c00d1 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:27.731856 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:27.740589 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479f5c79-606d-4689-929b-9f6b9ca52e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:27.741205 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f5c79-606d-4689-929b-9f6b9ca52e82 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:27.741482 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f5c79-606d-4689-929b-9f6b9ca52e82 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:27.741755 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f5c79-606d-4689-929b-9f6b9ca52e82 None admin] PATH_INFO: `/v3/services` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:27.742499 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479f5c79-606d-4689-929b-9f6b9ca52e82 None admin] RBAC: Adding query filter params (type=network) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:27.742801 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479f5c79-606d-4689-929b-9f6b9ca52e82 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:27.744206 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479f5c79-606d-4689-929b-9f6b9ca52e82 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:27.750502 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 295/589] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Oct 12 00:02:27 2022] GET /identity/v3/services?type=network => generated 350 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:27.751478 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479f5c79-606d-4689-929b-9f6b9ca52e82 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:27.790007 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f47ef6-d403-468e-be17-6ed3bc6106bc None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:27.826447 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ee9c7e-94ca-4c71-9c68-59d46d4678c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:27.827089 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ee9c7e-94ca-4c71-9c68-59d46d4678c3 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:27.827386 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ee9c7e-94ca-4c71-9c68-59d46d4678c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:27.827728 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ee9c7e-94ca-4c71-9c68-59d46d4678c3 None admin] PATH_INFO: `/v3/services` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:27.828343 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ee9c7e-94ca-4c71-9c68-59d46d4678c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:27.830336 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ee9c7e-94ca-4c71-9c68-59d46d4678c3 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:27.836270 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 295/590] 10.222.0.36 () {56 vars in 1087 bytes} [Wed Oct 12 00:02:27 2022] GET /identity/v3/services => generated 772 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:27.837076 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ee9c7e-94ca-4c71-9c68-59d46d4678c3 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:27.854179 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:27.869107 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479f5c79-606d-4689-929b-9f6b9ca52e82 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:27.953872 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b253ccb-171b-41a3-a56b-c40d009b409b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:27.954418 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b253ccb-171b-41a3-a56b-c40d009b409b None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:27.954696 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b253ccb-171b-41a3-a56b-c40d009b409b None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:27.955020 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b253ccb-171b-41a3-a56b-c40d009b409b None admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:27.955652 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ee9c7e-94ca-4c71-9c68-59d46d4678c3 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:27.955785 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b253ccb-171b-41a3-a56b-c40d009b409b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:27.956023 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b253ccb-171b-41a3-a56b-c40d009b409b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:27.957433 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b253ccb-171b-41a3-a56b-c40d009b409b None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:27.983033 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 296/591] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Oct 12 00:02:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:27.999563 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b253ccb-171b-41a3-a56b-c40d009b409b None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:28.006149 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8052f28-6ff7-4c93-a272-100fdc52487c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:28.006787 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8052f28-6ff7-4c93-a272-100fdc52487c None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:28.007064 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8052f28-6ff7-4c93-a272-100fdc52487c None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:28.007345 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8052f28-6ff7-4c93-a272-100fdc52487c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:28.008970 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8052f28-6ff7-4c93-a272-100fdc52487c None admin] RBAC: Adding query filter params (interface=public, service_id=af93554901e44b3a8e3e0166db5b6026, region_id=RegionOne) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:28.009291 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8052f28-6ff7-4c93-a272-100fdc52487c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:28.011218 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8052f28-6ff7-4c93-a272-100fdc52487c None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:28.029972 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 296/592] 10.222.0.36 () {56 vars in 1251 bytes} [Wed Oct 12 00:02:27 2022] GET /identity/v3/endpoints?service_id=af93554901e44b3a8e3e0166db5b6026&interface=public®ion_id=RegionOne => generated 194 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:28.035064 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8052f28-6ff7-4c93-a272-100fdc52487c None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:28.096320 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:28.129129 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:28.140552 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b253ccb-171b-41a3-a56b-c40d009b409b None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:28.156358 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8052f28-6ff7-4c93-a272-100fdc52487c None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:28.194168 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663afc60-4546-4fb4-a63a-0179c21ab09c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:28.195124 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663afc60-4546-4fb4-a63a-0179c21ab09c None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:28.195639 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663afc60-4546-4fb4-a63a-0179c21ab09c None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:28.196526 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663afc60-4546-4fb4-a63a-0179c21ab09c None admin] PATH_INFO: `/v3/services` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:28.197653 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663afc60-4546-4fb4-a63a-0179c21ab09c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:28.199534 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663afc60-4546-4fb4-a63a-0179c21ab09c None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:28.206594 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 297/593] 10.222.0.36 () {56 vars in 1087 bytes} [Wed Oct 12 00:02:28 2022] GET /identity/v3/services => generated 772 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 00:02:28.215242 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad97f6a2-4706-4d0f-8cf8-24fdacd932b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:28.220560 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad97f6a2-4706-4d0f-8cf8-24fdacd932b5 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:28.220560 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad97f6a2-4706-4d0f-8cf8-24fdacd932b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:28.220560 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad97f6a2-4706-4d0f-8cf8-24fdacd932b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:28.230402 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad97f6a2-4706-4d0f-8cf8-24fdacd932b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:28.232374 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad97f6a2-4706-4d0f-8cf8-24fdacd932b5 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:28.248607 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-ad97f6a2-4706-4d0f-8cf8-24fdacd932b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:28.248607 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:28.248607 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:28.248607 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:28.248607 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:28.248607 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:28.248607 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:28.248607 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:28.248607 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:28.248607 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:28.248607 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:28.248607 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:28.248607 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 00:02:28.248607 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 00:02:28.248607 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 00:02:28.248607 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 00:02:28.259108 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:28.259108 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:28.259108 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:28.259108 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:28.259108 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:28.259108 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:28.259108 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:28.259108 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:28.259108 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:28.259108 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:28.259108 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:28.259108 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:28.259108 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:28.259108 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:28.259108 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:28.259108 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:28.259108 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 00:02:28.260083 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 00:02:28.260083 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:28.260083 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:28.260083 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 297/594] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:02:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:28.260320 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663afc60-4546-4fb4-a63a-0179c21ab09c None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:28.411827 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663afc60-4546-4fb4-a63a-0179c21ab09c None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:28.464212 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745d1c2d-3891-4950-afa0-476e206a6874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:28.464765 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745d1c2d-3891-4950-afa0-476e206a6874 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:28.465043 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745d1c2d-3891-4950-afa0-476e206a6874 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:28.465310 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745d1c2d-3891-4950-afa0-476e206a6874 None admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:28.466005 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745d1c2d-3891-4950-afa0-476e206a6874 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:28.466315 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745d1c2d-3891-4950-afa0-476e206a6874 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:28.467846 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745d1c2d-3891-4950-afa0-476e206a6874 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:28.481348 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 298/595] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Oct 12 00:02:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:28.483724 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:28.487110 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad97f6a2-4706-4d0f-8cf8-24fdacd932b5 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:28.598865 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad97f6a2-4706-4d0f-8cf8-24fdacd932b5 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:28.607875 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:28.642976 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd16f8f0-0dac-4426-9f57-1ee8c11a162f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:28.643638 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd16f8f0-0dac-4426-9f57-1ee8c11a162f None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:28.643922 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd16f8f0-0dac-4426-9f57-1ee8c11a162f None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:28.644309 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd16f8f0-0dac-4426-9f57-1ee8c11a162f None admin] PATH_INFO: `/v3/users/swift` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:28.656083 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd16f8f0-0dac-4426-9f57-1ee8c11a162f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:28.657932 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd16f8f0-0dac-4426-9f57-1ee8c11a162f None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:28.671462 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-dd16f8f0-0dac-4426-9f57-1ee8c11a162f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 12 00:02:28.671462 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:28.671462 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:28.671462 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:28.671462 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:28.671462 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:28.671462 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:28.671462 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:28.671462 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:28.671462 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:28.671462 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:28.671462 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:28.671462 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 00:02:28.671462 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 00:02:28.671462 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 00:02:28.671462 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 00:02:28.672425 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:28.672425 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:28.672425 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 00:02:28.672425 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 00:02:28.672425 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 00:02:28.672425 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 00:02:28.672425 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:28.672425 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:28.672425 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:28.672425 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:28.672425 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:28.672425 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:28.672425 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:28.672425 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:28.672425 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:28.672425 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:28.672425 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:28.672425 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:28.673366 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:28.673366 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:28.673366 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 00:02:28.673366 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 00:02:28.673366 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 00:02:28.673366 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 00:02:28.673366 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 00:02:28.673366 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 00:02:28.673366 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 12 00:02:28.673366 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:28.673366 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 298/596] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Oct 12 00:02:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:28.678803 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745d1c2d-3891-4950-afa0-476e206a6874 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:28.839168 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745d1c2d-3891-4950-afa0-476e206a6874 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:28.889478 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a82909d-fa74-4e45-9f1c-2813cce1fe1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:28.890484 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a82909d-fa74-4e45-9f1c-2813cce1fe1b None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:28.890959 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a82909d-fa74-4e45-9f1c-2813cce1fe1b None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:28.891460 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a82909d-fa74-4e45-9f1c-2813cce1fe1b None admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:28.892441 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a82909d-fa74-4e45-9f1c-2813cce1fe1b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:28.892956 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a82909d-fa74-4e45-9f1c-2813cce1fe1b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:28.894597 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a82909d-fa74-4e45-9f1c-2813cce1fe1b None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:28.919487 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 299/597] 10.222.0.36 () {56 vars in 1135 bytes} [Wed Oct 12 00:02:28 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:28.925659 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd16f8f0-0dac-4426-9f57-1ee8c11a162f None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:29.067801 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd16f8f0-0dac-4426-9f57-1ee8c11a162f None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:29.093039 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffb5401-f35b-4d31-a55b-33ed5b6b0bd3 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:29.093733 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffb5401-f35b-4d31-a55b-33ed5b6b0bd3 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:29.094213 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffb5401-f35b-4d31-a55b-33ed5b6b0bd3 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:29.096662 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 300/598] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:02:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:02:29.102034 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e615b12-d792-4427-9674-51b0099b7d08 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:29.102327 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e615b12-d792-4427-9674-51b0099b7d08 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:29.102705 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e615b12-d792-4427-9674-51b0099b7d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:29.104443 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0776142-f77a-4642-95c5-9bb83f1eb7b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:29.105054 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0776142-f77a-4642-95c5-9bb83f1eb7b9 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:29.105334 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0776142-f77a-4642-95c5-9bb83f1eb7b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:29.105604 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0776142-f77a-4642-95c5-9bb83f1eb7b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:29.147328 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-7e615b12-d792-4427-9674-51b0099b7d08 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:02:29.148557 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-7e615b12-d792-4427-9674-51b0099b7d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:02:29.160859 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:29.175571 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0776142-f77a-4642-95c5-9bb83f1eb7b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:29.177030 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0776142-f77a-4642-95c5-9bb83f1eb7b9 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:29.280951 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 299/599] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Oct 12 00:02:28 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 12 00:02:29.288068 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0776142-f77a-4642-95c5-9bb83f1eb7b9 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:29.315319 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 301/600] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:02:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 00:02:29.316292 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:29.375844 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:29.406495 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0776142-f77a-4642-95c5-9bb83f1eb7b9 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:29.441577 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db52809-115f-4d52-a54f-b2e84cdba97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:29.442236 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db52809-115f-4d52-a54f-b2e84cdba97a None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:29.442516 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db52809-115f-4d52-a54f-b2e84cdba97a None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:29.442790 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db52809-115f-4d52-a54f-b2e84cdba97a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:29.456149 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db52809-115f-4d52-a54f-b2e84cdba97a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:29.457625 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db52809-115f-4d52-a54f-b2e84cdba97a None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:29.473789 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-2db52809-115f-4d52-a54f-b2e84cdba97a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:29.473789 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:29.473789 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:29.473789 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:29.473789 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:29.473789 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:29.473789 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:29.473789 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:29.473789 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:29.473789 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:29.473789 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:29.473789 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:29.473789 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 00:02:29.473789 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 00:02:29.473789 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 00:02:29.473789 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 00:02:29.474739 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:29.474739 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:29.474739 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:29.474739 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:29.474739 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:29.474739 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:29.474739 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:29.474739 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:29.474739 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:29.474739 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:29.474739 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:29.474739 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:29.474739 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:29.474739 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:29.474739 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:29.474739 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:29.474739 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 00:02:29.475652 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 00:02:29.475652 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:29.475652 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:29.475652 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 300/601] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:02:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:29.480488 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e615b12-d792-4427-9674-51b0099b7d08 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:29.621164 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d2ce9-36b4-4beb-b454-9afaf7e24d57 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:29.621488 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d2ce9-36b4-4beb-b454-9afaf7e24d57 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:29.621706 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d2ce9-36b4-4beb-b454-9afaf7e24d57 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:29.623190 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 301/602] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:02:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 00:02:29.628036 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06179ac2-96a0-4746-b364-f13e2f2b234e None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:29.628555 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06179ac2-96a0-4746-b364-f13e2f2b234e None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:29.628879 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e615b12-d792-4427-9674-51b0099b7d08 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:29.629250 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06179ac2-96a0-4746-b364-f13e2f2b234e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:29.664321 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-06179ac2-96a0-4746-b364-f13e2f2b234e None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:02:29.665494 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2247cf74-f5ee-453b-bc15-5fb95aea6e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:29.666109 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2247cf74-f5ee-453b-bc15-5fb95aea6e3b None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:29.666366 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-06179ac2-96a0-4746-b364-f13e2f2b234e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:02:29.666470 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2247cf74-f5ee-453b-bc15-5fb95aea6e3b None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:29.666708 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2247cf74-f5ee-453b-bc15-5fb95aea6e3b None admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:29.667353 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2247cf74-f5ee-453b-bc15-5fb95aea6e3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:29.667680 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2247cf74-f5ee-453b-bc15-5fb95aea6e3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:29.669065 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2247cf74-f5ee-453b-bc15-5fb95aea6e3b None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:29.683000 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 302/603] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Oct 12 00:02:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:29.692491 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2247cf74-f5ee-453b-bc15-5fb95aea6e3b None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:29.767678 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:29.776820 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 302/604] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:02:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 00:02:29.783525 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06179ac2-96a0-4746-b364-f13e2f2b234e None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:29.863460 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2247cf74-f5ee-453b-bc15-5fb95aea6e3b None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:29.866297 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:29.916533 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47cffbb-b946-47eb-ac29-12060b69145d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:29.916943 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06179ac2-96a0-4746-b364-f13e2f2b234e None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:29.917129 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47cffbb-b946-47eb-ac29-12060b69145d None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:29.917363 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47cffbb-b946-47eb-ac29-12060b69145d None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:29.917632 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47cffbb-b946-47eb-ac29-12060b69145d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:29.936528 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47cffbb-b946-47eb-ac29-12060b69145d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:29.937980 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47cffbb-b946-47eb-ac29-12060b69145d None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:29.971952 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-f47cffbb-b946-47eb-ac29-12060b69145d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 00:02:29.971952 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:29.971952 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:29.971952 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:29.971952 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:29.971952 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:29.971952 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:29.971952 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:29.971952 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:29.971952 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:29.971952 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:29.971952 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:29.971952 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 00:02:29.971952 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 00:02:29.971952 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 00:02:29.972884 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 00:02:29.972884 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:29.972884 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:29.972884 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:29.972884 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:29.972884 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:29.972884 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:29.972884 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:29.972884 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:29.972884 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:29.972884 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:29.972884 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:29.972884 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:29.972884 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:29.972884 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:29.972884 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:29.972884 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:29.973785 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 00:02:29.973785 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 00:02:29.973785 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 00:02:29.973785 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 00:02:29.973785 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 00:02:29.973785 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 00:02:29.973785 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 00:02:29.973785 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:29.973785 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 303/605] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Oct 12 00:02:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:29.980140 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42aa463-0f70-41b8-9615-f7e4cf057c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:29.980794 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42aa463-0f70-41b8-9615-f7e4cf057c1b None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:29.981077 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42aa463-0f70-41b8-9615-f7e4cf057c1b None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:29.981348 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42aa463-0f70-41b8-9615-f7e4cf057c1b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:29.981935 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42aa463-0f70-41b8-9615-f7e4cf057c1b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:29.983311 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42aa463-0f70-41b8-9615-f7e4cf057c1b None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:29.984480 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47cffbb-b946-47eb-ac29-12060b69145d None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:30.004651 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-d42aa463-0f70-41b8-9615-f7e4cf057c1b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 12 00:02:30.004651 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:30.004651 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 00:02:30.004651 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 00:02:30.004651 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 00:02:30.004651 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 00:02:30.004651 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 00:02:30.004651 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 00:02:30.004651 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 12 00:02:30.004651 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:30.004651 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 00:02:30.004651 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:30.004651 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:30.004651 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:30.004651 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:30.004651 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:30.005676 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:30.005676 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:30.005676 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:30.005676 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:30.005676 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:30.005676 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:30.005676 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:30.005676 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 00:02:30.005676 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 00:02:30.005676 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 00:02:30.005676 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 00:02:30.005676 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:30.005676 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:30.005676 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:30.005676 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:30.005676 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:30.005676 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:30.006558 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:30.006558 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:30.006558 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:30.006558 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:30.006558 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:30.006558 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:30.006558 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:30.006558 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:30.006558 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:30.006558 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:30.006558 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 00:02:30.006558 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 00:02:30.006558 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 12 00:02:30.006558 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:30.011787 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 303/606] 10.222.0.36 () {56 vars in 1132 bytes} [Wed Oct 12 00:02:29 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:30.017074 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42aa463-0f70-41b8-9615-f7e4cf057c1b None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:30.141041 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47cffbb-b946-47eb-ac29-12060b69145d None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:30.152396 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:30.173924 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42aa463-0f70-41b8-9615-f7e4cf057c1b None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:30.178330 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f1e0a4-0288-4735-8825-c329b6bc242f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:30.178508 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f1e0a4-0288-4735-8825-c329b6bc242f None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:30.178701 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f1e0a4-0288-4735-8825-c329b6bc242f None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:30.183117 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f1e0a4-0288-4735-8825-c329b6bc242f None admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:30.183117 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f1e0a4-0288-4735-8825-c329b6bc242f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:30.183117 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f1e0a4-0288-4735-8825-c329b6bc242f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:30.183117 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f1e0a4-0288-4735-8825-c329b6bc242f None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:30.192418 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:30.199034 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 304/607] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Oct 12 00:02:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:30.200097 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f1e0a4-0288-4735-8825-c329b6bc242f None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:30.218477 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c898ed-6dca-426d-b92a-a377dbc3f28e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:30.219691 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c898ed-6dca-426d-b92a-a377dbc3f28e None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:30.220153 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c898ed-6dca-426d-b92a-a377dbc3f28e None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:30.221412 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c898ed-6dca-426d-b92a-a377dbc3f28e None admin] PATH_INFO: `/v3/services` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:30.222625 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c898ed-6dca-426d-b92a-a377dbc3f28e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:30.223119 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c898ed-6dca-426d-b92a-a377dbc3f28e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:30.224694 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c898ed-6dca-426d-b92a-a377dbc3f28e None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:30.233153 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 304/608] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Oct 12 00:02:30 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 00:02:30.234545 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c898ed-6dca-426d-b92a-a377dbc3f28e None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:30.311156 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f1e0a4-0288-4735-8825-c329b6bc242f None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:30.351854 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4465cae-9cdb-41c5-ac38-be8cc6de63bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:30.352522 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4465cae-9cdb-41c5-ac38-be8cc6de63bb None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:30.353111 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4465cae-9cdb-41c5-ac38-be8cc6de63bb None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:30.353241 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4465cae-9cdb-41c5-ac38-be8cc6de63bb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:30.364575 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4465cae-9cdb-41c5-ac38-be8cc6de63bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 00:02:30.369588 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-e4465cae-9cdb-41c5-ac38-be8cc6de63bb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 12 00:02:30.369588 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:30.369588 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:30.369588 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:30.369588 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:30.369588 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:30.369588 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:30.369588 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:30.369588 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:30.369588 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:30.369588 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:30.369588 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:30.369588 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 00:02:30.369588 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 00:02:30.369588 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 00:02:30.369588 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 00:02:30.370584 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 00:02:30.370584 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 00:02:30.370584 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 00:02:30.370584 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 00:02:30.370584 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:30.370584 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:30.370584 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:30.370584 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:30.370584 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:30.370584 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:30.370584 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:30.370584 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:30.370584 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:30.370584 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:30.370584 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:30.370584 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:30.371442 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:30.371442 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:30.371442 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:30.371442 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:30.371442 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 00:02:30.371442 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 00:02:30.371442 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 00:02:30.371442 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 00:02:30.371442 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 00:02:30.371442 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 00:02:30.371442 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 12 00:02:30.371442 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:30.377653 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 305/609] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Oct 12 00:02:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:30.377653 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4465cae-9cdb-41c5-ac38-be8cc6de63bb None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:30.380863 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c898ed-6dca-426d-b92a-a377dbc3f28e None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:30.412585 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:30.417401 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3607cea-f4cb-442e-9b7d-c8a92d2c919a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:30.417980 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3607cea-f4cb-442e-9b7d-c8a92d2c919a None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:30.418332 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3607cea-f4cb-442e-9b7d-c8a92d2c919a None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:30.418689 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3607cea-f4cb-442e-9b7d-c8a92d2c919a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:30.419514 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3607cea-f4cb-442e-9b7d-c8a92d2c919a None admin] RBAC: Adding query filter params (role.id=b2e41f0d7a304dc7af8b6a232049b6a2, scope.project.id=0697927c6aac43d4b14a750fbbb1145e, user.id=fc1bcffc78844dcbb1a9b7c80285249b) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:30.419911 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3607cea-f4cb-442e-9b7d-c8a92d2c919a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:30.421487 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3607cea-f4cb-442e-9b7d-c8a92d2c919a None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:30.435649 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 305/610] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Oct 12 00:02:30 2022] GET /identity/v3/role_assignments?user.id=fc1bcffc78844dcbb1a9b7c80285249b&scope.project.id=0697927c6aac43d4b14a750fbbb1145e&role.id=b2e41f0d7a304dc7af8b6a232049b6a2 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:30.437010 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3607cea-f4cb-442e-9b7d-c8a92d2c919a None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:30.507490 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4465cae-9cdb-41c5-ac38-be8cc6de63bb None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:30.532818 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:30.567994 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db4f7ce-50c6-46b6-93be-5e8565d78cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:30.569029 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db4f7ce-50c6-46b6-93be-5e8565d78cb0 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:30.569532 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db4f7ce-50c6-46b6-93be-5e8565d78cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:30.570014 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db4f7ce-50c6-46b6-93be-5e8565d78cb0 None admin] PATH_INFO: `/v3/services` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:30.570862 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db4f7ce-50c6-46b6-93be-5e8565d78cb0 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:30.572377 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db4f7ce-50c6-46b6-93be-5e8565d78cb0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:30.574321 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db4f7ce-50c6-46b6-93be-5e8565d78cb0 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:30.601488 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 306/611] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Oct 12 00:02:30 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:30.639947 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3607cea-f4cb-442e-9b7d-c8a92d2c919a None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:30.649457 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fe8ee1-5dce-4234-93e6-26bb1655c2c5 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:30.650085 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fe8ee1-5dce-4234-93e6-26bb1655c2c5 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:30.650604 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fe8ee1-5dce-4234-93e6-26bb1655c2c5 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:30.652262 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 307/612] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:02:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 00:02:30.659201 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5854a569-45fd-4a93-acb6-3372b62ec91c None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:30.660459 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5854a569-45fd-4a93-acb6-3372b62ec91c None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:30.660968 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5854a569-45fd-4a93-acb6-3372b62ec91c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:30.684699 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-5854a569-45fd-4a93-acb6-3372b62ec91c None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:02:30.686701 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-5854a569-45fd-4a93-acb6-3372b62ec91c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:02:30.719763 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65083b1-55e4-410e-bb71-3b3840948d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:30.720447 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65083b1-55e4-410e-bb71-3b3840948d4c None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:30.720735 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65083b1-55e4-410e-bb71-3b3840948d4c None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:30.721009 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65083b1-55e4-410e-bb71-3b3840948d4c None admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:30.721693 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65083b1-55e4-410e-bb71-3b3840948d4c None admin] RBAC: Adding query filter params (name=service) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:30.722007 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65083b1-55e4-410e-bb71-3b3840948d4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:30.723629 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65083b1-55e4-410e-bb71-3b3840948d4c None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:30.733806 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 306/613] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Oct 12 00:02:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:30.742247 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65083b1-55e4-410e-bb71-3b3840948d4c None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:30.787785 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 308/614] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:02:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 00:02:30.815475 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727456a2-a825-450c-af16-8256055b0bbb None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:30.816086 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727456a2-a825-450c-af16-8256055b0bbb None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:30.816588 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727456a2-a825-450c-af16-8256055b0bbb None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:30.818155 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 309/615] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:02:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 00:02:30.823248 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e56122d-11fb-4e38-a47f-d8f442fd5471 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:30.823631 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e56122d-11fb-4e38-a47f-d8f442fd5471 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:30.823867 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e56122d-11fb-4e38-a47f-d8f442fd5471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:30.848030 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-2e56122d-11fb-4e38-a47f-d8f442fd5471 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:02:30.850113 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-2e56122d-11fb-4e38-a47f-d8f442fd5471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:02:30.862181 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65083b1-55e4-410e-bb71-3b3840948d4c None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:30.863909 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:30.892568 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:30.903405 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e1f011-beae-458f-8f50-79d3bb9881df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:30.904059 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e1f011-beae-458f-8f50-79d3bb9881df None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:30.904335 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e1f011-beae-458f-8f50-79d3bb9881df None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:30.904604 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e1f011-beae-458f-8f50-79d3bb9881df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:30.915642 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e1f011-beae-458f-8f50-79d3bb9881df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:30.917110 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e1f011-beae-458f-8f50-79d3bb9881df None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:30.933678 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-30e1f011-beae-458f-8f50-79d3bb9881df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:30.933678 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:30.933678 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:30.933678 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:30.933678 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:30.933678 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:30.933678 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:30.933678 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:30.933678 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:30.933678 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:30.933678 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:30.933678 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:30.933678 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 00:02:30.933678 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 00:02:30.933678 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 00:02:30.933678 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 00:02:30.934695 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:30.934695 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:30.934695 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:30.934695 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:30.934695 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:30.934695 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:30.934695 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:30.934695 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:30.934695 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:30.934695 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:30.934695 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:30.934695 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:30.934695 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:30.934695 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:30.934695 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:30.934695 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:30.934695 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 00:02:30.936116 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 00:02:30.936116 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:30.936116 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:30.936116 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 307/616] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:02:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:30.943509 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e1f011-beae-458f-8f50-79d3bb9881df None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:30.953498 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 310/617] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:02:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 12 00:02:31.058410 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e1f011-beae-458f-8f50-79d3bb9881df None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:31.096143 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534c9e0c-7340-4189-bc1c-992edde85c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:31.097166 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534c9e0c-7340-4189-bc1c-992edde85c1a None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:31.097648 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534c9e0c-7340-4189-bc1c-992edde85c1a None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:31.098145 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534c9e0c-7340-4189-bc1c-992edde85c1a None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:31.098984 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534c9e0c-7340-4189-bc1c-992edde85c1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:31.099538 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534c9e0c-7340-4189-bc1c-992edde85c1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:31.101157 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534c9e0c-7340-4189-bc1c-992edde85c1a None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:31.115247 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 308/618] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Oct 12 00:02:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:31.122754 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e56122d-11fb-4e38-a47f-d8f442fd5471 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:31.160998 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534c9e0c-7340-4189-bc1c-992edde85c1a None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:31.180535 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:31.296410 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e56122d-11fb-4e38-a47f-d8f442fd5471 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:31.316324 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534c9e0c-7340-4189-bc1c-992edde85c1a None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:31.346233 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f57827-ef29-486b-9f05-f5e32c492f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:31.347264 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f57827-ef29-486b-9f05-f5e32c492f4f None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:31.347566 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f57827-ef29-486b-9f05-f5e32c492f4f None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:31.348088 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f57827-ef29-486b-9f05-f5e32c492f4f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:31.354632 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7a6fdb-9d9d-405b-8a47-cfb4cfa7589b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:31.355336 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7a6fdb-9d9d-405b-8a47-cfb4cfa7589b None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:31.355799 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7a6fdb-9d9d-405b-8a47-cfb4cfa7589b None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:31.356140 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7a6fdb-9d9d-405b-8a47-cfb4cfa7589b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:31.356250 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f57827-ef29-486b-9f05-f5e32c492f4f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:31.356847 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7a6fdb-9d9d-405b-8a47-cfb4cfa7589b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:31.357700 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f57827-ef29-486b-9f05-f5e32c492f4f None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:31.358337 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7a6fdb-9d9d-405b-8a47-cfb4cfa7589b None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:31.368753 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-ea7a6fdb-9d9d-405b-8a47-cfb4cfa7589b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 12 00:02:31.368753 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:31.368753 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 00:02:31.368753 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 00:02:31.368753 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 00:02:31.368753 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 00:02:31.368753 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 00:02:31.368753 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 00:02:31.368753 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 12 00:02:31.368753 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:31.368753 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 00:02:31.368753 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:31.368753 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:31.368753 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:31.368753 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:31.368753 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:31.368753 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:31.369996 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:31.369996 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:31.369996 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:31.369996 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:31.369996 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:31.369996 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:31.369996 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 00:02:31.369996 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 00:02:31.369996 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 00:02:31.369996 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 00:02:31.369996 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:31.369996 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:31.369996 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:31.369996 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:31.369996 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:31.369996 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:31.371056 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:31.371056 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:31.371056 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:31.371056 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:31.371056 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:31.371056 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:31.371056 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:31.371056 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:31.371056 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:31.371056 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:31.371056 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 00:02:31.371056 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 00:02:31.371056 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 12 00:02:31.371056 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:31.373151 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-75f57827-ef29-486b-9f05-f5e32c492f4f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 12 00:02:31.373151 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:31.373151 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:31.373151 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:31.373151 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:31.373151 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:31.373151 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:31.373151 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:31.373151 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:31.373151 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:31.373151 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:31.373151 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:31.373151 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 00:02:31.373151 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 00:02:31.373151 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 00:02:31.373151 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 00:02:31.375199 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:31.375199 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:31.375199 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 00:02:31.375199 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 00:02:31.375199 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 00:02:31.375199 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 00:02:31.375199 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:31.375199 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:31.375199 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:31.375199 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:31.375199 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:31.375199 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:31.375199 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:31.375199 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:31.375199 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:31.375199 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:31.375199 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:31.375199 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:31.377104 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:31.377104 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:31.377104 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 00:02:31.377104 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 00:02:31.377104 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 00:02:31.377104 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 00:02:31.377104 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 00:02:31.377104 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 00:02:31.377104 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 12 00:02:31.377104 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:31.377833 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 309/619] 10.222.0.36 () {56 vars in 1129 bytes} [Wed Oct 12 00:02:31 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 00:02:31.377833 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7a6fdb-9d9d-405b-8a47-cfb4cfa7589b None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:31.378018 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 311/620] 10.222.0.36 () {56 vars in 1117 bytes} [Wed Oct 12 00:02:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:31.378018 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f57827-ef29-486b-9f05-f5e32c492f4f None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:31.396100 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:31.474538 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:31.498117 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7a6fdb-9d9d-405b-8a47-cfb4cfa7589b None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:31.505466 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f57827-ef29-486b-9f05-f5e32c492f4f None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:31.544604 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d389eb63-25c6-48f4-bf2b-24fa0d89952f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:31.545173 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d389eb63-25c6-48f4-bf2b-24fa0d89952f None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:31.545447 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d389eb63-25c6-48f4-bf2b-24fa0d89952f None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:31.545713 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d389eb63-25c6-48f4-bf2b-24fa0d89952f None admin] PATH_INFO: `/v3/services/network` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:31.546302 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d389eb63-25c6-48f4-bf2b-24fa0d89952f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:31.547749 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d389eb63-25c6-48f4-bf2b-24fa0d89952f None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:31.549270 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709c4f1a-2213-46c7-8a15-919e9eacd4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:31.549877 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709c4f1a-2213-46c7-8a15-919e9eacd4dd None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:31.550148 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709c4f1a-2213-46c7-8a15-919e9eacd4dd None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:31.550409 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709c4f1a-2213-46c7-8a15-919e9eacd4dd None admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:31.551130 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709c4f1a-2213-46c7-8a15-919e9eacd4dd None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:31.551452 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709c4f1a-2213-46c7-8a15-919e9eacd4dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:31.552924 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709c4f1a-2213-46c7-8a15-919e9eacd4dd None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:31.562547 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-d389eb63-25c6-48f4-bf2b-24fa0d89952f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 12 00:02:31.562547 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:31.562547 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 00:02:31.562547 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 00:02:31.562547 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 00:02:31.562547 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 00:02:31.562547 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 00:02:31.562547 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 00:02:31.562547 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 12 00:02:31.562547 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:31.562547 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 00:02:31.562547 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:31.562547 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:31.562547 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:31.562547 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:31.562547 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:31.562547 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:31.563832 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:31.563832 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:31.563832 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:31.563832 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:31.563832 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:31.563832 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:31.563832 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 00:02:31.563832 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 00:02:31.563832 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 00:02:31.563832 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 00:02:31.563832 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:31.563832 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:31.563832 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:31.563832 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:31.563832 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:31.563832 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:31.564701 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:31.564701 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:31.564701 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:31.564701 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:31.564701 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:31.564701 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:31.564701 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:31.564701 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:31.564701 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:31.564701 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:31.564701 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 00:02:31.564701 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 00:02:31.564701 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 12 00:02:31.564701 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:31.567456 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 310/621] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Oct 12 00:02:31 2022] GET /identity/v3/services/network => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:31.568502 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d389eb63-25c6-48f4-bf2b-24fa0d89952f None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:31.582210 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 312/622] 10.222.0.36 () {56 vars in 1149 bytes} [Wed Oct 12 00:02:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:31.583178 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709c4f1a-2213-46c7-8a15-919e9eacd4dd None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:31.657309 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:31.690795 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709c4f1a-2213-46c7-8a15-919e9eacd4dd None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:31.695281 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d389eb63-25c6-48f4-bf2b-24fa0d89952f None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:31.727651 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b4be6a-2d71-4015-be81-d30ec0873617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:31.728312 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b4be6a-2d71-4015-be81-d30ec0873617 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:31.728639 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b4be6a-2d71-4015-be81-d30ec0873617 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:31.728907 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b4be6a-2d71-4015-be81-d30ec0873617 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:31.737946 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b2aded-eb21-42c7-80f5-6ed3bff26023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:31.738808 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b2aded-eb21-42c7-80f5-6ed3bff26023 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:31.739287 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b2aded-eb21-42c7-80f5-6ed3bff26023 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:31.739821 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b2aded-eb21-42c7-80f5-6ed3bff26023 None admin] PATH_INFO: `/v3/services` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:31.740645 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b2aded-eb21-42c7-80f5-6ed3bff26023 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:31.741162 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b2aded-eb21-42c7-80f5-6ed3bff26023 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:31.742739 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b2aded-eb21-42c7-80f5-6ed3bff26023 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:31.753134 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 311/623] 10.222.0.36 () {56 vars in 1124 bytes} [Wed Oct 12 00:02:31 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:31.754375 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b2aded-eb21-42c7-80f5-6ed3bff26023 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:31.764794 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:31.800957 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b4be6a-2d71-4015-be81-d30ec0873617 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:31.802349 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b4be6a-2d71-4015-be81-d30ec0873617 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:31.871310 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:31.877309 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b2aded-eb21-42c7-80f5-6ed3bff26023 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:31.903127 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 313/624] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Oct 12 00:02:31 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 12 00:02:31.908987 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b4be6a-2d71-4015-be81-d30ec0873617 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:31.916438 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3ea395-debc-4661-b14e-48550611861b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:31.917341 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3ea395-debc-4661-b14e-48550611861b None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:31.917824 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3ea395-debc-4661-b14e-48550611861b None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:31.918320 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3ea395-debc-4661-b14e-48550611861b None admin] PATH_INFO: `/v3/services` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:31.919160 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3ea395-debc-4661-b14e-48550611861b None admin] RBAC: Adding query filter params (name=network) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:31.919687 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3ea395-debc-4661-b14e-48550611861b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:31.921288 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3ea395-debc-4661-b14e-48550611861b None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:31.927915 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 312/625] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Oct 12 00:02:31 2022] GET /identity/v3/services?name=network => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:31.929014 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3ea395-debc-4661-b14e-48550611861b None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:32.009724 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:32.016979 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b4be6a-2d71-4015-be81-d30ec0873617 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:32.048190 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3ea395-debc-4661-b14e-48550611861b None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:32.054311 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0cf512-756e-4a80-9313-168ebcc31fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:32.054868 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0cf512-756e-4a80-9313-168ebcc31fac None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:32.055149 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0cf512-756e-4a80-9313-168ebcc31fac None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:32.055455 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0cf512-756e-4a80-9313-168ebcc31fac None admin] PATH_INFO: `/v3/services` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:32.056143 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0cf512-756e-4a80-9313-168ebcc31fac None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:32.056448 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0cf512-756e-4a80-9313-168ebcc31fac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:32.057814 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0cf512-756e-4a80-9313-168ebcc31fac None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:32.067718 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 314/626] 10.222.0.36 () {56 vars in 1124 bytes} [Wed Oct 12 00:02:31 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:32.067718 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0cf512-756e-4a80-9313-168ebcc31fac None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:32.099127 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0bb1b5-ba8d-4aff-8e6b-6341faebe42a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:32.099982 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0bb1b5-ba8d-4aff-8e6b-6341faebe42a None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:32.100262 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0bb1b5-ba8d-4aff-8e6b-6341faebe42a None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:32.100532 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0bb1b5-ba8d-4aff-8e6b-6341faebe42a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:32.107790 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:32.113536 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0bb1b5-ba8d-4aff-8e6b-6341faebe42a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:32.115530 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0bb1b5-ba8d-4aff-8e6b-6341faebe42a None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:32.132526 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-ea0bb1b5-ba8d-4aff-8e6b-6341faebe42a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:32.132526 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:32.132526 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:32.132526 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:32.132526 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:32.132526 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:32.132526 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:32.132526 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:32.132526 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:32.132526 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:32.132526 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:32.132526 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:32.132526 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 00:02:32.132526 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 00:02:32.132526 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 00:02:32.132526 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 00:02:32.133818 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:32.133818 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:32.133818 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:32.133818 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:32.133818 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:32.133818 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:32.133818 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:32.133818 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:32.133818 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:32.133818 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:32.133818 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:32.133818 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:32.133818 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:32.133818 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:32.133818 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:32.133818 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:32.133818 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 00:02:32.134716 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 00:02:32.134716 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:32.134716 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:32.138688 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 313/627] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:02:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:32.139857 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0bb1b5-ba8d-4aff-8e6b-6341faebe42a None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:32.183699 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0cf512-756e-4a80-9313-168ebcc31fac None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:32.219213 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67aa444-f672-415e-b237-14eb3dc0219f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:32.220166 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67aa444-f672-415e-b237-14eb3dc0219f None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:32.221139 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67aa444-f672-415e-b237-14eb3dc0219f None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:32.221575 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67aa444-f672-415e-b237-14eb3dc0219f None admin] PATH_INFO: `/v3/services` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:32.223984 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67aa444-f672-415e-b237-14eb3dc0219f None admin] RBAC: Adding query filter params (type=network) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:32.223984 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67aa444-f672-415e-b237-14eb3dc0219f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:32.224402 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67aa444-f672-415e-b237-14eb3dc0219f None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:32.229978 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 315/628] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Oct 12 00:02:32 2022] GET /identity/v3/services?type=network => generated 350 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:32.230957 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67aa444-f672-415e-b237-14eb3dc0219f None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:32.257076 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0bb1b5-ba8d-4aff-8e6b-6341faebe42a None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:32.284372 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:32.297926 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c4d24b-f6a2-4ff8-9894-cd213b311c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:32.298564 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c4d24b-f6a2-4ff8-9894-cd213b311c46 None admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:32.299079 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c4d24b-f6a2-4ff8-9894-cd213b311c46 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:32.299216 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c4d24b-f6a2-4ff8-9894-cd213b311c46 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:32.299814 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c4d24b-f6a2-4ff8-9894-cd213b311c46 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:32.301170 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c4d24b-f6a2-4ff8-9894-cd213b311c46 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:32.336838 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:32.341291 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67aa444-f672-415e-b237-14eb3dc0219f None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:32.374794 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962317d2-ec12-45d6-9915-f68588e48841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:32.375476 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962317d2-ec12-45d6-9915-f68588e48841 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:32.375726 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962317d2-ec12-45d6-9915-f68588e48841 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:32.375943 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962317d2-ec12-45d6-9915-f68588e48841 None admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:32.376575 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962317d2-ec12-45d6-9915-f68588e48841 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:32.376867 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962317d2-ec12-45d6-9915-f68588e48841 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:32.378209 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962317d2-ec12-45d6-9915-f68588e48841 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:32.395241 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 316/629] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Oct 12 00:02:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:32.395241 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962317d2-ec12-45d6-9915-f68588e48841 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:32.427928 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 314/630] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:02:32 2022] POST /identity/v3/endpoints => generated 346 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:02:32.428980 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c4d24b-f6a2-4ff8-9894-cd213b311c46 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:32.578988 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962317d2-ec12-45d6-9915-f68588e48841 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:32.625644 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12573787-d55a-49f0-bae4-d524628e537e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:32.627155 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12573787-d55a-49f0-bae4-d524628e537e None admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:32.627683 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12573787-d55a-49f0-bae4-d524628e537e None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:32.628186 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12573787-d55a-49f0-bae4-d524628e537e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:32.629079 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12573787-d55a-49f0-bae4-d524628e537e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:32.630843 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12573787-d55a-49f0-bae4-d524628e537e None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:32.664091 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c4d24b-f6a2-4ff8-9894-cd213b311c46 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:32.701859 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91648aa-0e08-4d8c-aaaa-7d3e988dfff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:32.702886 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91648aa-0e08-4d8c-aaaa-7d3e988dfff6 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:32.703389 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91648aa-0e08-4d8c-aaaa-7d3e988dfff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:32.703909 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91648aa-0e08-4d8c-aaaa-7d3e988dfff6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:32.712710 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91648aa-0e08-4d8c-aaaa-7d3e988dfff6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:32.714159 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91648aa-0e08-4d8c-aaaa-7d3e988dfff6 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:32.719262 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 317/631] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:02:32 2022] POST /identity/v3/endpoints => generated 337 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:02:32.729760 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-a91648aa-0e08-4d8c-aaaa-7d3e988dfff6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 00:02:32.729760 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:32.729760 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:32.729760 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:32.729760 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:32.729760 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:32.729760 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:32.729760 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:32.729760 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:32.729760 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:32.729760 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:32.729760 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:32.729760 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 00:02:32.729760 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 00:02:32.729760 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 00:02:32.731194 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 00:02:32.731194 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:32.731194 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:32.731194 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:32.731194 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:32.731194 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:32.731194 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:32.731194 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:32.731194 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:32.731194 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:32.731194 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:32.731194 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:32.731194 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:32.731194 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:32.731194 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:32.731194 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:32.731194 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:32.732190 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 00:02:32.732190 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 00:02:32.732190 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 00:02:32.732190 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 00:02:32.732190 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 00:02:32.732190 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 00:02:32.732190 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 00:02:32.732190 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:32.732190 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 315/632] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Oct 12 00:02:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:32.740736 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12573787-d55a-49f0-bae4-d524628e537e None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:32.757111 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:32.874425 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12573787-d55a-49f0-bae4-d524628e537e None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:32.915484 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959d5c0e-6771-4b2c-ab2b-af9094f56d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:32.916485 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959d5c0e-6771-4b2c-ab2b-af9094f56d1f None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:32.917024 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959d5c0e-6771-4b2c-ab2b-af9094f56d1f None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:32.917502 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959d5c0e-6771-4b2c-ab2b-af9094f56d1f None admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:32.918424 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959d5c0e-6771-4b2c-ab2b-af9094f56d1f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:32.918946 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959d5c0e-6771-4b2c-ab2b-af9094f56d1f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:32.920650 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959d5c0e-6771-4b2c-ab2b-af9094f56d1f None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:32.934826 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 318/633] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Oct 12 00:02:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:32.938036 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d54b616-0215-41dd-b1bc-ce0ddb56b270 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:32.938036 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d54b616-0215-41dd-b1bc-ce0ddb56b270 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:32.938036 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d54b616-0215-41dd-b1bc-ce0ddb56b270 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:32.939479 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 316/634] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:02:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:02:32.941750 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959d5c0e-6771-4b2c-ab2b-af9094f56d1f None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:32.945000 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794d62ba-ff86-42cc-9b7b-664c06e644c6 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:32.945254 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794d62ba-ff86-42cc-9b7b-664c06e644c6 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:32.945452 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794d62ba-ff86-42cc-9b7b-664c06e644c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:32.966834 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-794d62ba-ff86-42cc-9b7b-664c06e644c6 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:02:32.968212 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-794d62ba-ff86-42cc-9b7b-664c06e644c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:02:33.010479 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:33.036431 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:33.065958 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 317/635] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:02:32 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 12 00:02:33.081964 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959d5c0e-6771-4b2c-ab2b-af9094f56d1f None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:33.088032 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924c5947-9d61-42f4-a492-2889b0a3a681 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:33.088032 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924c5947-9d61-42f4-a492-2889b0a3a681 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:33.088277 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924c5947-9d61-42f4-a492-2889b0a3a681 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:33.093324 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 318/636] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:02:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 00:02:33.096080 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f153622-4405-4424-8175-ede62702f8a9 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:33.096354 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f153622-4405-4424-8175-ede62702f8a9 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:33.096744 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f153622-4405-4424-8175-ede62702f8a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:33.119318 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-4f153622-4405-4424-8175-ede62702f8a9 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:02:33.120643 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-4f153622-4405-4424-8175-ede62702f8a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:02:33.128167 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596c9f2e-5dd1-4768-ba0f-058335a0dc57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:33.129542 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596c9f2e-5dd1-4768-ba0f-058335a0dc57 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:33.130061 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596c9f2e-5dd1-4768-ba0f-058335a0dc57 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:33.130545 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596c9f2e-5dd1-4768-ba0f-058335a0dc57 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:33.131511 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596c9f2e-5dd1-4768-ba0f-058335a0dc57 None admin] RBAC: Adding query filter params (role.id=a53c0c4b8b8645b6a1cfe96266daec8e, scope.project.id=0697927c6aac43d4b14a750fbbb1145e, user.id=d030d18b595144f19d6ad01d4cd5d52b) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:33.132417 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596c9f2e-5dd1-4768-ba0f-058335a0dc57 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:33.135071 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596c9f2e-5dd1-4768-ba0f-058335a0dc57 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:33.143095 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 319/637] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Oct 12 00:02:32 2022] GET /identity/v3/role_assignments?user.id=d030d18b595144f19d6ad01d4cd5d52b&scope.project.id=0697927c6aac43d4b14a750fbbb1145e&role.id=a53c0c4b8b8645b6a1cfe96266daec8e => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:33.207827 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 319/638] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:02:33 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 12 00:02:33.364562 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596c9f2e-5dd1-4768-ba0f-058335a0dc57 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:33.465900 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596c9f2e-5dd1-4768-ba0f-058335a0dc57 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:33.492353 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f153622-4405-4424-8175-ede62702f8a9 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:33.496319 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:33.502645 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b47e1d-d6bd-4eb0-a758-9495ea9f90b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:33.503323 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b47e1d-d6bd-4eb0-a758-9495ea9f90b4 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:33.503633 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b47e1d-d6bd-4eb0-a758-9495ea9f90b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:33.503904 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b47e1d-d6bd-4eb0-a758-9495ea9f90b4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:33.504489 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b47e1d-d6bd-4eb0-a758-9495ea9f90b4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:33.505918 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b47e1d-d6bd-4eb0-a758-9495ea9f90b4 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:33.515130 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-53b47e1d-d6bd-4eb0-a758-9495ea9f90b4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 12 00:02:33.515130 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:33.515130 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 00:02:33.515130 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 00:02:33.515130 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 00:02:33.515130 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 00:02:33.515130 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 00:02:33.515130 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 00:02:33.515130 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 12 00:02:33.515130 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:33.515130 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 00:02:33.515130 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:33.515130 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:33.515130 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:33.515130 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:33.515130 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:33.515130 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:33.516242 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:33.516242 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:33.516242 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:33.516242 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:33.516242 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:33.516242 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:33.516242 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 00:02:33.516242 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 00:02:33.516242 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 00:02:33.516242 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 00:02:33.516242 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:33.516242 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:33.516242 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:33.516242 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:33.516242 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:33.516242 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:33.517773 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:33.517773 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:33.517773 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:33.517773 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:33.517773 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:33.517773 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:33.517773 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:33.517773 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:33.517773 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:33.517773 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:33.517773 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 00:02:33.517773 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 00:02:33.517773 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 12 00:02:33.517773 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:33.517773 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 320/639] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Oct 12 00:02:33 2022] GET /identity/v3/services/object-store => generated 93 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:33.522389 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b47e1d-d6bd-4eb0-a758-9495ea9f90b4 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:33.618605 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:33.623515 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f153622-4405-4424-8175-ede62702f8a9 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:33.634398 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b47e1d-d6bd-4eb0-a758-9495ea9f90b4 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:33.662509 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca51e75-6739-419d-80f2-562ae062c855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:33.663598 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca51e75-6739-419d-80f2-562ae062c855 None admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:33.664104 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca51e75-6739-419d-80f2-562ae062c855 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:33.664633 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca51e75-6739-419d-80f2-562ae062c855 None admin] PATH_INFO: `/v3/services` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:33.665587 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca51e75-6739-419d-80f2-562ae062c855 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:33.667274 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca51e75-6739-419d-80f2-562ae062c855 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:33.671310 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb81312f-f3cf-43a3-b01a-1e80341c5b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:33.671855 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb81312f-f3cf-43a3-b01a-1e80341c5b4b None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:33.672136 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb81312f-f3cf-43a3-b01a-1e80341c5b4b None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:33.672410 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb81312f-f3cf-43a3-b01a-1e80341c5b4b None admin] PATH_INFO: `/v3/services` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:33.673044 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb81312f-f3cf-43a3-b01a-1e80341c5b4b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:33.673381 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb81312f-f3cf-43a3-b01a-1e80341c5b4b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:33.674749 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb81312f-f3cf-43a3-b01a-1e80341c5b4b None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:33.679875 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 321/640] 10.222.0.36 () {56 vars in 1122 bytes} [Wed Oct 12 00:02:33 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 00:02:33.685189 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb81312f-f3cf-43a3-b01a-1e80341c5b4b None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:33.748207 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:33.772441 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 320/641] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Oct 12 00:02:33 2022] POST /identity/v3/services => generated 269 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 00:02:33.802104 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb81312f-f3cf-43a3-b01a-1e80341c5b4b None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:33.850064 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f48747-5394-4667-9c19-58c6366b8272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:33.851094 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f48747-5394-4667-9c19-58c6366b8272 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:33.851606 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f48747-5394-4667-9c19-58c6366b8272 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:33.852121 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f48747-5394-4667-9c19-58c6366b8272 None admin] PATH_INFO: `/v3/services` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:33.853055 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f48747-5394-4667-9c19-58c6366b8272 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:33.853634 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f48747-5394-4667-9c19-58c6366b8272 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:33.855247 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f48747-5394-4667-9c19-58c6366b8272 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:33.861302 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 322/642] 10.222.0.36 () {56 vars in 1122 bytes} [Wed Oct 12 00:02:33 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 00:02:34.870180 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f406a383-c610-4765-aeb9-de8a0849d7a0 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:34.870962 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f406a383-c610-4765-aeb9-de8a0849d7a0 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:34.870962 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f406a383-c610-4765-aeb9-de8a0849d7a0 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:34.872395 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 321/643] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:02:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:02:34.878534 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784860be-4354-482b-88b4-d41ddb385745 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:34.878959 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784860be-4354-482b-88b4-d41ddb385745 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:34.879152 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784860be-4354-482b-88b4-d41ddb385745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:34.898199 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-784860be-4354-482b-88b4-d41ddb385745 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:02:34.899403 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-784860be-4354-482b-88b4-d41ddb385745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:02:34.975208 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 323/644] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:02:34 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 00:02:35.244195 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f406a383-c610-4765-aeb9-de8a0849d7a0 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:35.353634 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f406a383-c610-4765-aeb9-de8a0849d7a0 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:35.391787 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37775563-8166-40d0-8fc9-5ea0eaeca0d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:35.392427 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37775563-8166-40d0-8fc9-5ea0eaeca0d6 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:35.392706 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37775563-8166-40d0-8fc9-5ea0eaeca0d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:35.392977 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37775563-8166-40d0-8fc9-5ea0eaeca0d6 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:35.393559 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37775563-8166-40d0-8fc9-5ea0eaeca0d6 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:35.394926 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37775563-8166-40d0-8fc9-5ea0eaeca0d6 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:35.404227 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-37775563-8166-40d0-8fc9-5ea0eaeca0d6 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 12 00:02:35.404227 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:35.404227 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 00:02:35.404227 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 00:02:35.404227 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 00:02:35.404227 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 00:02:35.404227 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 00:02:35.404227 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 00:02:35.404227 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 12 00:02:35.404227 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:35.404227 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 00:02:35.404227 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:35.404227 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:35.404227 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:35.404227 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:35.404227 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:35.404227 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:35.405207 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:35.405207 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:35.405207 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:35.405207 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:35.405207 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:35.405207 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:35.405207 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 00:02:35.405207 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 00:02:35.405207 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 00:02:35.405207 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 00:02:35.405207 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:35.405207 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:35.405207 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:35.405207 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:35.405207 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:35.405207 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:35.406014 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:35.406014 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:35.406014 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:35.406014 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:35.406014 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:35.406014 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:35.406014 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:35.406014 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:35.406014 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:35.406014 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:35.406014 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 00:02:35.406014 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 00:02:35.406014 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 12 00:02:35.406014 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:35.407496 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 322/645] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Oct 12 00:02:35 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:35.411582 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37775563-8166-40d0-8fc9-5ea0eaeca0d6 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:35.412175 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d846f-3ab0-4bf3-8fb0-475c61450c02 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:35.412570 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d846f-3ab0-4bf3-8fb0-475c61450c02 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:35.412959 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d846f-3ab0-4bf3-8fb0-475c61450c02 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:35.414668 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 324/646] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:02:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:02:35.419943 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e6137f-382e-47db-9b14-85cb87f100fd None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:35.420346 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e6137f-382e-47db-9b14-85cb87f100fd None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:35.420726 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e6137f-382e-47db-9b14-85cb87f100fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:35.440074 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-a7e6137f-382e-47db-9b14-85cb87f100fd None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:02:35.440617 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:35.441489 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-a7e6137f-382e-47db-9b14-85cb87f100fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:02:35.499833 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:35.519702 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 325/647] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:02:35 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 12 00:02:35.546692 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37775563-8166-40d0-8fc9-5ea0eaeca0d6 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:35.582481 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd03d5f-eb77-4483-96f4-f204f289a39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:35.583166 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd03d5f-eb77-4483-96f4-f204f289a39e None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:35.583478 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd03d5f-eb77-4483-96f4-f204f289a39e None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:35.583755 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd03d5f-eb77-4483-96f4-f204f289a39e None admin] PATH_INFO: `/v3/services` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:35.584381 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd03d5f-eb77-4483-96f4-f204f289a39e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:35.584684 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd03d5f-eb77-4483-96f4-f204f289a39e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:35.586048 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd03d5f-eb77-4483-96f4-f204f289a39e None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:35.595572 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 323/648] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Oct 12 00:02:35 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:35.597346 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e6137f-382e-47db-9b14-85cb87f100fd None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:35.703991 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e6137f-382e-47db-9b14-85cb87f100fd None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:35.740827 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5141bcf7-0802-4954-9bf3-81e8717c673d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:35.741446 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5141bcf7-0802-4954-9bf3-81e8717c673d None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:35.741727 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5141bcf7-0802-4954-9bf3-81e8717c673d None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:35.742000 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5141bcf7-0802-4954-9bf3-81e8717c673d None admin] PATH_INFO: `/v3/services` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:35.742633 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5141bcf7-0802-4954-9bf3-81e8717c673d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:35.742934 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5141bcf7-0802-4954-9bf3-81e8717c673d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:35.744333 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5141bcf7-0802-4954-9bf3-81e8717c673d None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:35.755549 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 326/649] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Oct 12 00:02:35 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:35.794257 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd03d5f-eb77-4483-96f4-f204f289a39e None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:35.893043 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:35.911572 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd03d5f-eb77-4483-96f4-f204f289a39e None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:35.954131 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf538fed-a24c-419a-9623-06ec6dc92182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:35.954820 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf538fed-a24c-419a-9623-06ec6dc92182 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:35.955099 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf538fed-a24c-419a-9623-06ec6dc92182 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:35.955394 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf538fed-a24c-419a-9623-06ec6dc92182 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:35.965928 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf538fed-a24c-419a-9623-06ec6dc92182 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 00:02:35.970927 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-cf538fed-a24c-419a-9623-06ec6dc92182 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 12 00:02:35.970927 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:35.970927 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:35.970927 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:35.970927 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:35.970927 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:35.970927 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:35.970927 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:35.970927 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:35.970927 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:35.970927 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:35.970927 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:35.970927 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 00:02:35.970927 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 00:02:35.970927 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 00:02:35.970927 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 00:02:35.971895 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 00:02:35.971895 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 00:02:35.971895 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 00:02:35.971895 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 00:02:35.971895 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:35.971895 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:35.971895 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:35.971895 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:35.971895 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:35.971895 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:35.971895 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:35.971895 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:35.971895 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:35.971895 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:35.971895 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:35.971895 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:35.972767 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:35.972767 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:35.972767 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:35.972767 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:35.972767 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 00:02:35.972767 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 00:02:35.972767 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 00:02:35.972767 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 00:02:35.972767 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 00:02:35.972767 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 00:02:35.972767 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 12 00:02:35.972767 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:35.972767 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 324/650] 10.222.0.36 () {56 vars in 1120 bytes} [Wed Oct 12 00:02:35 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:35.981087 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5141bcf7-0802-4954-9bf3-81e8717c673d None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:36.013585 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a68ec2-b885-41c3-8d54-8c767d80ac5a None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:36.014025 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a68ec2-b885-41c3-8d54-8c767d80ac5a None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:36.014426 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a68ec2-b885-41c3-8d54-8c767d80ac5a None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:36.016530 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 325/651] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:02:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:02:36.020945 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1286c004-8bbf-4fc2-b118-e0e485a348d3 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:36.021225 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1286c004-8bbf-4fc2-b118-e0e485a348d3 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:36.021498 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1286c004-8bbf-4fc2-b118-e0e485a348d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:36.032603 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:36.040089 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-1286c004-8bbf-4fc2-b118-e0e485a348d3 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:02:36.041278 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-1286c004-8bbf-4fc2-b118-e0e485a348d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:02:36.099996 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5141bcf7-0802-4954-9bf3-81e8717c673d None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:36.114008 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 326/652] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:02:36 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 12 00:02:36.116180 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d2807f-cb57-4296-8a88-cb536a0db381 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:36.116936 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d2807f-cb57-4296-8a88-cb536a0db381 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:36.117427 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d2807f-cb57-4296-8a88-cb536a0db381 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:36.118881 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 327/653] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:02:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:02:36.124453 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33495917-3a11-41f9-aab1-87abf4634208 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:36.124738 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33495917-3a11-41f9-aab1-87abf4634208 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:36.125018 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33495917-3a11-41f9-aab1-87abf4634208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:36.145625 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c50bf3e-d84a-4864-8d23-054d01d333ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:36.146170 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-33495917-3a11-41f9-aab1-87abf4634208 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:02:36.146867 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c50bf3e-d84a-4864-8d23-054d01d333ab None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:36.147394 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-33495917-3a11-41f9-aab1-87abf4634208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:02:36.147615 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c50bf3e-d84a-4864-8d23-054d01d333ab None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:36.148122 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c50bf3e-d84a-4864-8d23-054d01d333ab None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:36.149026 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c50bf3e-d84a-4864-8d23-054d01d333ab None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:36.149551 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c50bf3e-d84a-4864-8d23-054d01d333ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:36.151169 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c50bf3e-d84a-4864-8d23-054d01d333ab None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:36.160506 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 327/654] 10.222.0.36 () {56 vars in 1115 bytes} [Wed Oct 12 00:02:35 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 00:02:36.166325 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c50bf3e-d84a-4864-8d23-054d01d333ab None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:36.222015 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 328/655] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:02:36 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 00:02:36.286388 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c50bf3e-d84a-4864-8d23-054d01d333ab None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:36.288991 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:36.322154 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3718a8-49c7-4b77-a0e3-d437c0d54c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:36.323122 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3718a8-49c7-4b77-a0e3-d437c0d54c7c None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:36.323693 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3718a8-49c7-4b77-a0e3-d437c0d54c7c None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:36.324381 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3718a8-49c7-4b77-a0e3-d437c0d54c7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:36.334566 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3718a8-49c7-4b77-a0e3-d437c0d54c7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:36.336191 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3718a8-49c7-4b77-a0e3-d437c0d54c7c None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:36.351959 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-1a3718a8-49c7-4b77-a0e3-d437c0d54c7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:36.351959 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:36.351959 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:36.351959 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:36.351959 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:36.351959 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:36.351959 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:36.351959 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:36.351959 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:36.351959 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:36.351959 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:36.351959 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:36.351959 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 00:02:36.351959 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 00:02:36.351959 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 00:02:36.351959 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 00:02:36.353145 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:36.353145 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:36.353145 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:36.353145 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:36.353145 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:36.353145 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:36.353145 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:36.353145 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:36.353145 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:36.353145 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:36.353145 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:36.353145 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:36.353145 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:36.353145 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:36.353145 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:36.353145 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:36.353145 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 00:02:36.354268 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 00:02:36.354268 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:36.354268 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:36.355992 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 328/656] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:02:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:36.362269 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33495917-3a11-41f9-aab1-87abf4634208 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:36.371245 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:36.403533 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3718a8-49c7-4b77-a0e3-d437c0d54c7c None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:36.476821 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33495917-3a11-41f9-aab1-87abf4634208 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:36.515598 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879c8cf4-ae9e-47ba-a8fa-33099b3204ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:36.516215 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879c8cf4-ae9e-47ba-a8fa-33099b3204ed None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:36.516489 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879c8cf4-ae9e-47ba-a8fa-33099b3204ed None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:36.517373 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879c8cf4-ae9e-47ba-a8fa-33099b3204ed None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:36.518018 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879c8cf4-ae9e-47ba-a8fa-33099b3204ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:36.518318 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879c8cf4-ae9e-47ba-a8fa-33099b3204ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:36.519704 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879c8cf4-ae9e-47ba-a8fa-33099b3204ed None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:36.519822 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3718a8-49c7-4b77-a0e3-d437c0d54c7c None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:36.532710 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 329/657] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Oct 12 00:02:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:36.538297 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879c8cf4-ae9e-47ba-a8fa-33099b3204ed None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:36.559627 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952a6703-7b40-4f8d-867b-bda385703331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:36.560266 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952a6703-7b40-4f8d-867b-bda385703331 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:36.560540 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952a6703-7b40-4f8d-867b-bda385703331 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:36.560806 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952a6703-7b40-4f8d-867b-bda385703331 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:36.561382 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952a6703-7b40-4f8d-867b-bda385703331 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:36.562737 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952a6703-7b40-4f8d-867b-bda385703331 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:36.572352 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-952a6703-7b40-4f8d-867b-bda385703331 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 12 00:02:36.572352 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:36.572352 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 00:02:36.572352 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 00:02:36.572352 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 00:02:36.572352 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 00:02:36.572352 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 00:02:36.572352 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 00:02:36.572352 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 12 00:02:36.572352 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:36.572352 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 00:02:36.572352 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:36.572352 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:36.572352 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:36.572352 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:36.572352 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:36.573348 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:36.573348 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:36.573348 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:36.573348 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:36.573348 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:36.573348 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:36.573348 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:36.573348 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 00:02:36.573348 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 00:02:36.573348 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 00:02:36.573348 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 00:02:36.573348 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:36.573348 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:36.573348 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:36.573348 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:36.573348 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:36.573348 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:36.574242 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:36.574242 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:36.574242 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:36.574242 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:36.574242 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:36.574242 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:36.574242 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:36.574242 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:36.574242 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:36.574242 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:36.574242 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 00:02:36.574242 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 00:02:36.574242 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 12 00:02:36.574242 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:36.574242 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 329/658] 10.222.0.36 () {56 vars in 1132 bytes} [Wed Oct 12 00:02:36 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:36.575068 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952a6703-7b40-4f8d-867b-bda385703331 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:36.597350 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:36.621678 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:36.658944 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879c8cf4-ae9e-47ba-a8fa-33099b3204ed None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:36.678123 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952a6703-7b40-4f8d-867b-bda385703331 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:36.697007 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404eb543-37e7-478c-8e08-8ac5e2f4c962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:36.697630 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404eb543-37e7-478c-8e08-8ac5e2f4c962 None admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:36.697903 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404eb543-37e7-478c-8e08-8ac5e2f4c962 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:36.698171 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404eb543-37e7-478c-8e08-8ac5e2f4c962 None admin] PATH_INFO: `/v3/services` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:36.698812 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404eb543-37e7-478c-8e08-8ac5e2f4c962 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:36.700207 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404eb543-37e7-478c-8e08-8ac5e2f4c962 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:36.712201 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de11d858-9971-4f09-8845-4a023e2d8259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:36.712833 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de11d858-9971-4f09-8845-4a023e2d8259 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:36.715615 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de11d858-9971-4f09-8845-4a023e2d8259 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:36.715615 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de11d858-9971-4f09-8845-4a023e2d8259 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:36.720997 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de11d858-9971-4f09-8845-4a023e2d8259 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:36.722479 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de11d858-9971-4f09-8845-4a023e2d8259 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:36.735186 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-de11d858-9971-4f09-8845-4a023e2d8259 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 12 00:02:36.735186 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:36.735186 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:36.735186 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:36.735186 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:36.735186 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:36.735186 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:36.735186 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:36.735186 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:36.735186 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:36.735186 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:36.735186 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:36.735186 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 00:02:36.735186 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 00:02:36.735186 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 00:02:36.735186 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 00:02:36.736952 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:36.736952 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:36.736952 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 00:02:36.736952 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 00:02:36.736952 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 00:02:36.736952 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 00:02:36.736952 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:36.736952 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:36.736952 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:36.736952 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:36.736952 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:36.736952 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:36.736952 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:36.736952 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:36.736952 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:36.736952 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:36.736952 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:36.736952 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:36.737966 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:36.737966 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:36.737966 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 00:02:36.737966 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 00:02:36.737966 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 00:02:36.737966 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 00:02:36.737966 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 00:02:36.737966 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 00:02:36.737966 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 12 00:02:36.737966 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:36.737966 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 330/659] 10.222.0.36 () {56 vars in 1117 bytes} [Wed Oct 12 00:02:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:36.738714 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de11d858-9971-4f09-8845-4a023e2d8259 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:36.808922 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 330/660] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Oct 12 00:02:36 2022] POST /identity/v3/services => generated 241 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:02:36.809953 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404eb543-37e7-478c-8e08-8ac5e2f4c962 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:36.834178 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:36.856997 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de11d858-9971-4f09-8845-4a023e2d8259 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:36.909692 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cd2814-13d2-46d2-9e9f-50a9c1b9a40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:36.910720 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd2814-13d2-46d2-9e9f-50a9c1b9a40f None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:36.911207 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd2814-13d2-46d2-9e9f-50a9c1b9a40f None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:36.911717 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd2814-13d2-46d2-9e9f-50a9c1b9a40f None admin] PATH_INFO: `/v3/services` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:36.912569 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cd2814-13d2-46d2-9e9f-50a9c1b9a40f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:36.913136 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cd2814-13d2-46d2-9e9f-50a9c1b9a40f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:36.914803 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cd2814-13d2-46d2-9e9f-50a9c1b9a40f None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:36.925727 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 331/661] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Oct 12 00:02:36 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:36.927031 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cd2814-13d2-46d2-9e9f-50a9c1b9a40f None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:36.956516 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404eb543-37e7-478c-8e08-8ac5e2f4c962 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:36.987843 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:36.995745 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f95d71b-1f93-4152-b631-00b90ea5d15a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:36.996358 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f95d71b-1f93-4152-b631-00b90ea5d15a None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:36.996631 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f95d71b-1f93-4152-b631-00b90ea5d15a None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:36.999582 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f95d71b-1f93-4152-b631-00b90ea5d15a None admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:36.999582 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f95d71b-1f93-4152-b631-00b90ea5d15a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:36.999582 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f95d71b-1f93-4152-b631-00b90ea5d15a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:36.999582 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f95d71b-1f93-4152-b631-00b90ea5d15a None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:37.020132 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 331/662] 10.222.0.36 () {56 vars in 1149 bytes} [Wed Oct 12 00:02:36 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:37.026556 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f95d71b-1f93-4152-b631-00b90ea5d15a None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:37.052624 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cd2814-13d2-46d2-9e9f-50a9c1b9a40f None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:37.092700 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1f2672-cf4c-4f97-bc8b-8a4460c49efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:37.093326 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1f2672-cf4c-4f97-bc8b-8a4460c49efc None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:37.093606 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1f2672-cf4c-4f97-bc8b-8a4460c49efc None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:37.093877 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1f2672-cf4c-4f97-bc8b-8a4460c49efc None admin] PATH_INFO: `/v3/services` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:37.094681 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1f2672-cf4c-4f97-bc8b-8a4460c49efc None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:37.095565 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1f2672-cf4c-4f97-bc8b-8a4460c49efc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:37.097621 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1f2672-cf4c-4f97-bc8b-8a4460c49efc None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:37.103583 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 332/663] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Oct 12 00:02:36 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:37.109289 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1f2672-cf4c-4f97-bc8b-8a4460c49efc None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:37.116015 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:37.162599 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f95d71b-1f93-4152-b631-00b90ea5d15a None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:37.199231 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d0ffcc-19a8-4156-b5ae-66a0d9a4825b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:37.200001 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d0ffcc-19a8-4156-b5ae-66a0d9a4825b None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:37.200280 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d0ffcc-19a8-4156-b5ae-66a0d9a4825b None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:37.200557 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d0ffcc-19a8-4156-b5ae-66a0d9a4825b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:37.213137 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d0ffcc-19a8-4156-b5ae-66a0d9a4825b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:37.214906 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d0ffcc-19a8-4156-b5ae-66a0d9a4825b None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:37.224591 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1f2672-cf4c-4f97-bc8b-8a4460c49efc None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:37.232815 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-53d0ffcc-19a8-4156-b5ae-66a0d9a4825b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:37.232815 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:37.232815 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:37.232815 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:37.232815 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:37.232815 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:37.232815 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:37.232815 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:37.232815 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:37.232815 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:37.232815 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:37.232815 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:37.232815 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 00:02:37.232815 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 00:02:37.232815 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 00:02:37.232815 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 00:02:37.234016 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:37.234016 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:37.234016 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:37.234016 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:37.234016 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:37.234016 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:37.234016 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:37.234016 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:37.234016 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:37.234016 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:37.234016 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:37.234016 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:37.234016 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:37.234016 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:37.234016 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:37.234016 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:37.234016 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 00:02:37.235012 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 00:02:37.235012 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:37.235012 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:37.239732 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 332/664] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:02:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 00:02:37.240666 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d0ffcc-19a8-4156-b5ae-66a0d9a4825b None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:37.266741 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a164896-7f8f-4fa4-abca-b125191722d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:37.268295 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a164896-7f8f-4fa4-abca-b125191722d7 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:37.268770 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a164896-7f8f-4fa4-abca-b125191722d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:37.269210 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a164896-7f8f-4fa4-abca-b125191722d7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:37.270182 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a164896-7f8f-4fa4-abca-b125191722d7 None admin] RBAC: Adding query filter params (interface=public, service_id=471b195617c440a3b4e5b751ef7b5068, region_id=RegionOne) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:37.270659 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a164896-7f8f-4fa4-abca-b125191722d7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:37.272253 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a164896-7f8f-4fa4-abca-b125191722d7 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:37.278209 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 333/665] 10.222.0.36 () {56 vars in 1251 bytes} [Wed Oct 12 00:02:37 2022] GET /identity/v3/endpoints?service_id=471b195617c440a3b4e5b751ef7b5068&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:37.282886 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:37.283387 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a164896-7f8f-4fa4-abca-b125191722d7 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:37.360160 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d0ffcc-19a8-4156-b5ae-66a0d9a4825b None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:37.366010 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:37.404434 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a164896-7f8f-4fa4-abca-b125191722d7 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:37.404667 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97be2c3-554a-4ca5-a66e-ff40543b4fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:37.405273 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97be2c3-554a-4ca5-a66e-ff40543b4fe8 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:37.405556 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97be2c3-554a-4ca5-a66e-ff40543b4fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:37.405827 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97be2c3-554a-4ca5-a66e-ff40543b4fe8 None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:37.406473 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97be2c3-554a-4ca5-a66e-ff40543b4fe8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:37.406771 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97be2c3-554a-4ca5-a66e-ff40543b4fe8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:37.409526 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97be2c3-554a-4ca5-a66e-ff40543b4fe8 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:37.425145 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 333/666] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Oct 12 00:02:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:37.430565 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97be2c3-554a-4ca5-a66e-ff40543b4fe8 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:37.447420 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650f57e1-0d63-4c33-b541-23f0bfba2d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:37.448258 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650f57e1-0d63-4c33-b541-23f0bfba2d58 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:37.448781 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650f57e1-0d63-4c33-b541-23f0bfba2d58 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:37.449223 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650f57e1-0d63-4c33-b541-23f0bfba2d58 None admin] PATH_INFO: `/v3/services` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:37.449885 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650f57e1-0d63-4c33-b541-23f0bfba2d58 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:37.451609 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650f57e1-0d63-4c33-b541-23f0bfba2d58 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:37.459487 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 334/667] 10.222.0.36 () {56 vars in 1087 bytes} [Wed Oct 12 00:02:37 2022] GET /identity/v3/services => generated 1258 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 00:02:37.551473 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:37.558057 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97be2c3-554a-4ca5-a66e-ff40543b4fe8 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:37.596623 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148b1a5b-68f4-4281-803f-3c1fa7f80174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:37.597285 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148b1a5b-68f4-4281-803f-3c1fa7f80174 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:37.597566 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148b1a5b-68f4-4281-803f-3c1fa7f80174 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:37.597842 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148b1a5b-68f4-4281-803f-3c1fa7f80174 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:37.606590 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148b1a5b-68f4-4281-803f-3c1fa7f80174 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:37.608111 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148b1a5b-68f4-4281-803f-3c1fa7f80174 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:37.623494 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-148b1a5b-68f4-4281-803f-3c1fa7f80174 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 00:02:37.623494 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:37.623494 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:37.623494 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:37.623494 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:37.623494 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:37.623494 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:37.623494 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:37.623494 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:37.623494 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:37.623494 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:37.623494 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:37.623494 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 00:02:37.623494 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 00:02:37.623494 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 00:02:37.624437 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 00:02:37.624437 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:37.624437 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:37.624437 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:37.624437 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:37.624437 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:37.624437 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:37.624437 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:37.624437 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:37.624437 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:37.624437 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:37.624437 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:37.624437 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:37.624437 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:37.624437 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:37.624437 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:37.624437 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:37.625316 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 00:02:37.625316 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 00:02:37.625316 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 00:02:37.625316 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 00:02:37.625316 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 00:02:37.625316 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 00:02:37.625316 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 00:02:37.625316 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:37.625316 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 334/668] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Oct 12 00:02:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 00:02:37.632892 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650f57e1-0d63-4c33-b541-23f0bfba2d58 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:37.742319 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650f57e1-0d63-4c33-b541-23f0bfba2d58 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:37.773095 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:37.778594 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869deca5-1636-4716-a961-ae53f342d24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:37.779268 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869deca5-1636-4716-a961-ae53f342d24b None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:37.779582 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869deca5-1636-4716-a961-ae53f342d24b None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:37.779849 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869deca5-1636-4716-a961-ae53f342d24b None admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:37.780585 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869deca5-1636-4716-a961-ae53f342d24b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:37.780886 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869deca5-1636-4716-a961-ae53f342d24b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:37.782902 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869deca5-1636-4716-a961-ae53f342d24b None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:37.795434 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 335/669] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Oct 12 00:02:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:37.802060 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148b1a5b-68f4-4281-803f-3c1fa7f80174 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:37.910011 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148b1a5b-68f4-4281-803f-3c1fa7f80174 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:37.948260 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2a3055-1051-4d63-9c03-80d8f15955e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:37.948934 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2a3055-1051-4d63-9c03-80d8f15955e1 None admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:37.949212 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2a3055-1051-4d63-9c03-80d8f15955e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:37.949486 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2a3055-1051-4d63-9c03-80d8f15955e1 None admin] PATH_INFO: `/v3/projects/0697927c6aac43d4b14a750fbbb1145e/users/d030d18b595144f19d6ad01d4cd5d52b/roles/59b6984c4af34bffa59fdac6d215a6a3` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:37.951739 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2a3055-1051-4d63-9c03-80d8f15955e1 None admin] RBAC: Authorizing `identity:create_grant(project_id=0697927c6aac43d4b14a750fbbb1145e, user_id=d030d18b595144f19d6ad01d4cd5d52b, role_id=59b6984c4af34bffa59fdac6d215a6a3)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:37.953199 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2a3055-1051-4d63-9c03-80d8f15955e1 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:38.024543 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 335/670] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Oct 12 00:02:37 2022] PUT /identity/v3/projects/0697927c6aac43d4b14a750fbbb1145e/users/d030d18b595144f19d6ad01d4cd5d52b/roles/59b6984c4af34bffa59fdac6d215a6a3 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:02:38.044998 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbff1b8-f2db-472c-b537-6a321e220b46 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:38.045359 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbff1b8-f2db-472c-b537-6a321e220b46 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:38.045531 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbff1b8-f2db-472c-b537-6a321e220b46 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:38.047121 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 336/671] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:02:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:02:38.060038 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f38a8-f8b2-4ecb-980c-d8bfcb1ebfdd None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:38.060336 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f38a8-f8b2-4ecb-980c-d8bfcb1ebfdd None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:38.060736 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f38a8-f8b2-4ecb-980c-d8bfcb1ebfdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:38.082849 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-5c2f38a8-f8b2-4ecb-980c-d8bfcb1ebfdd None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:02:38.084169 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-5c2f38a8-f8b2-4ecb-980c-d8bfcb1ebfdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:02:38.098905 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:38.162541 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 336/672] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:02:38 2022] POST /identity/v3/auth/tokens => generated 1373 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 00:02:38.434513 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbff1b8-f2db-472c-b537-6a321e220b46 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:38.544209 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbff1b8-f2db-472c-b537-6a321e220b46 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:38.581090 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a85034-6661-4471-9c65-933c15fae8ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:38.582069 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a85034-6661-4471-9c65-933c15fae8ae None admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:38.582549 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a85034-6661-4471-9c65-933c15fae8ae None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:38.583026 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a85034-6661-4471-9c65-933c15fae8ae None admin] PATH_INFO: `/v3/services` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:38.583926 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a85034-6661-4471-9c65-933c15fae8ae None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:38.585536 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a85034-6661-4471-9c65-933c15fae8ae None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:38.682024 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 337/673] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Oct 12 00:02:38 2022] POST /identity/v3/services => generated 251 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:02:39.180647 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5257e4-d620-4ba2-8151-4e2b75c9e50c None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:39.181166 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5257e4-d620-4ba2-8151-4e2b75c9e50c None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:39.181166 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5257e4-d620-4ba2-8151-4e2b75c9e50c None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:39.182795 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 337/674] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:02:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:02:39.188598 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4073cd8-9e3c-4cf0-a7b0-03deb63db56a None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:39.188889 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4073cd8-9e3c-4cf0-a7b0-03deb63db56a None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:39.189166 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4073cd8-9e3c-4cf0-a7b0-03deb63db56a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:39.208399 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-d4073cd8-9e3c-4cf0-a7b0-03deb63db56a None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:02:39.209606 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-d4073cd8-9e3c-4cf0-a7b0-03deb63db56a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:02:39.261488 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:39.288390 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 338/675] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:02:39 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 00:02:39.569338 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5257e4-d620-4ba2-8151-4e2b75c9e50c None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:39.683693 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5257e4-d620-4ba2-8151-4e2b75c9e50c None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:39.718829 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d18be49-1335-4b70-aee0-fb2e853a4aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:39.719540 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d18be49-1335-4b70-aee0-fb2e853a4aa4 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:39.719853 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d18be49-1335-4b70-aee0-fb2e853a4aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:39.720118 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d18be49-1335-4b70-aee0-fb2e853a4aa4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:39.720692 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d18be49-1335-4b70-aee0-fb2e853a4aa4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:39.722108 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d18be49-1335-4b70-aee0-fb2e853a4aa4 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:39.723624 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4fc474-6d68-4c34-9e5c-f107ee92cddc None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:39.724064 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4fc474-6d68-4c34-9e5c-f107ee92cddc None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:39.724154 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4fc474-6d68-4c34-9e5c-f107ee92cddc None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:39.725735 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 339/676] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:02:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:02:39.730668 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e73c2-d3e4-4de1-af90-bab70e05448f None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:39.730951 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e73c2-d3e4-4de1-af90-bab70e05448f None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:39.731075 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-4d18be49-1335-4b70-aee0-fb2e853a4aa4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 12 00:02:39.731075 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:39.731075 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 00:02:39.731075 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 00:02:39.731075 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 00:02:39.731075 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 00:02:39.731075 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 00:02:39.731075 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 00:02:39.731075 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 12 00:02:39.731075 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:39.731075 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 00:02:39.731075 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:39.731075 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:39.731075 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:39.731075 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:39.731075 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:39.731075 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:39.732020 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:39.732020 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:39.732020 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:39.732020 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:39.732020 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:39.732020 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:39.732020 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 00:02:39.732020 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 00:02:39.732020 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 00:02:39.732020 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 00:02:39.732020 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:39.732020 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:39.732020 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:39.732020 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:39.732020 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:39.732020 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:39.733281 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:39.733281 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:39.733281 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:39.733281 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:39.733281 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:39.733281 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:39.733281 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:39.733281 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:39.733281 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:39.733281 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:39.733281 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 00:02:39.733281 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 00:02:39.733281 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 12 00:02:39.733281 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:39.734107 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e73c2-d3e4-4de1-af90-bab70e05448f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:39.734191 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 338/677] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Oct 12 00:02:39 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:39.740543 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d18be49-1335-4b70-aee0-fb2e853a4aa4 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:39.750337 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-d07e73c2-d3e4-4de1-af90-bab70e05448f None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:02:39.752249 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-d07e73c2-d3e4-4de1-af90-bab70e05448f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:02:39.755738 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:39.827713 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 340/678] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:02:39 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 12 00:02:39.862317 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d18be49-1335-4b70-aee0-fb2e853a4aa4 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:39.896976 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d881d130-0870-4fe5-b41f-6f195dbd2ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:39.897917 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d881d130-0870-4fe5-b41f-6f195dbd2ee0 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:39.898389 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d881d130-0870-4fe5-b41f-6f195dbd2ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:39.898899 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d881d130-0870-4fe5-b41f-6f195dbd2ee0 None admin] PATH_INFO: `/v3/services` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:39.899761 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d881d130-0870-4fe5-b41f-6f195dbd2ee0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:39.900255 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d881d130-0870-4fe5-b41f-6f195dbd2ee0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:39.901813 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d881d130-0870-4fe5-b41f-6f195dbd2ee0 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:39.907090 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 339/679] 10.222.0.36 () {56 vars in 1122 bytes} [Wed Oct 12 00:02:39 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:39.912440 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07e73c2-d3e4-4de1-af90-bab70e05448f None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:40.013697 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07e73c2-d3e4-4de1-af90-bab70e05448f None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:40.023350 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:40.048780 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93207dc3-09b7-42d6-8f8b-e53f87c45732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:40.049388 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93207dc3-09b7-42d6-8f8b-e53f87c45732 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:40.049680 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93207dc3-09b7-42d6-8f8b-e53f87c45732 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:40.049951 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93207dc3-09b7-42d6-8f8b-e53f87c45732 None admin] PATH_INFO: `/v3/services` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:40.050564 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93207dc3-09b7-42d6-8f8b-e53f87c45732 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:40.050857 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93207dc3-09b7-42d6-8f8b-e53f87c45732 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:40.052236 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93207dc3-09b7-42d6-8f8b-e53f87c45732 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:40.057257 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 341/680] 10.222.0.36 () {56 vars in 1122 bytes} [Wed Oct 12 00:02:39 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:40.062785 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d881d130-0870-4fe5-b41f-6f195dbd2ee0 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:40.095264 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93207dc3-09b7-42d6-8f8b-e53f87c45732 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:40.143942 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:40.175100 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d881d130-0870-4fe5-b41f-6f195dbd2ee0 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:40.214202 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208222fd-19e0-4fc2-abb9-2445723ae322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:40.214926 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208222fd-19e0-4fc2-abb9-2445723ae322 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:40.215207 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208222fd-19e0-4fc2-abb9-2445723ae322 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:40.215503 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208222fd-19e0-4fc2-abb9-2445723ae322 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:40.216520 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208222fd-19e0-4fc2-abb9-2445723ae322 None admin] RBAC: Adding query filter params (interface=public, service_id=133212c5ae9942d68601e37894bc1428, region_id=RegionOne) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:40.216830 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208222fd-19e0-4fc2-abb9-2445723ae322 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:40.218457 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208222fd-19e0-4fc2-abb9-2445723ae322 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:40.225646 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 340/681] 10.222.0.36 () {56 vars in 1251 bytes} [Wed Oct 12 00:02:40 2022] GET /identity/v3/endpoints?service_id=133212c5ae9942d68601e37894bc1428&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:40.226827 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93207dc3-09b7-42d6-8f8b-e53f87c45732 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:40.232057 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208222fd-19e0-4fc2-abb9-2445723ae322 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:40.271476 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2968d585-69d8-44f4-827c-dd8644bec2ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:40.272109 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2968d585-69d8-44f4-827c-dd8644bec2ed None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:40.272384 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2968d585-69d8-44f4-827c-dd8644bec2ed None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:40.272651 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2968d585-69d8-44f4-827c-dd8644bec2ed None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:40.273295 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2968d585-69d8-44f4-827c-dd8644bec2ed None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:40.274648 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2968d585-69d8-44f4-827c-dd8644bec2ed None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:40.284947 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-2968d585-69d8-44f4-827c-dd8644bec2ed None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 12 00:02:40.284947 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:40.284947 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 00:02:40.284947 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 00:02:40.284947 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 00:02:40.284947 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 00:02:40.284947 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 00:02:40.284947 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 00:02:40.284947 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 12 00:02:40.284947 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:40.284947 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 00:02:40.284947 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:40.284947 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:40.284947 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:40.284947 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:40.284947 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:40.286536 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:40.286536 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:40.286536 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:40.286536 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:40.286536 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:40.286536 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:40.286536 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:40.286536 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 00:02:40.286536 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 00:02:40.286536 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 00:02:40.286536 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 00:02:40.286536 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:40.286536 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:40.286536 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:40.286536 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:40.286536 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:40.286536 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:40.288118 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:40.288118 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:40.288118 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:40.288118 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:40.288118 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:40.288118 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:40.288118 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:40.288118 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:40.288118 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:40.288118 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:40.288118 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 00:02:40.288118 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 00:02:40.288118 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 12 00:02:40.288118 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:40.288118 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 342/682] 10.222.0.36 () {56 vars in 1132 bytes} [Wed Oct 12 00:02:40 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 00:02:40.288943 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c9aa92-1380-43f0-9269-2f63182b37a0 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:40.288943 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c9aa92-1380-43f0-9269-2f63182b37a0 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:40.288943 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c9aa92-1380-43f0-9269-2f63182b37a0 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:40.293944 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 343/683] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:02:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:02:40.294454 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c9aa92-1380-43f0-9269-2f63182b37a0 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:40.334625 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:40.350331 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208222fd-19e0-4fc2-abb9-2445723ae322 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:40.391084 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac37e9ed-b29c-4d71-8d56-3e1565c8e41b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:40.391963 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac37e9ed-b29c-4d71-8d56-3e1565c8e41b None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:40.392142 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac37e9ed-b29c-4d71-8d56-3e1565c8e41b None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:40.392402 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac37e9ed-b29c-4d71-8d56-3e1565c8e41b None admin] PATH_INFO: `/v3/services` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:40.393007 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac37e9ed-b29c-4d71-8d56-3e1565c8e41b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:40.394619 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac37e9ed-b29c-4d71-8d56-3e1565c8e41b None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:40.401558 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c9aa92-1380-43f0-9269-2f63182b37a0 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:40.407788 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 341/684] 10.222.0.36 () {56 vars in 1087 bytes} [Wed Oct 12 00:02:40 2022] GET /identity/v3/services => generated 1497 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:02:40.409635 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7487dd09-f9f6-4eda-9e30-36c0e2fdea4c None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:40.410910 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7487dd09-f9f6-4eda-9e30-36c0e2fdea4c None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:40.411466 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7487dd09-f9f6-4eda-9e30-36c0e2fdea4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:40.421080 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:40.440133 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-7487dd09-f9f6-4eda-9e30-36c0e2fdea4c None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:02:40.442214 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-7487dd09-f9f6-4eda-9e30-36c0e2fdea4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:02:40.445083 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f7936e-fa43-49b2-8568-609f736a93a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:40.445919 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f7936e-fa43-49b2-8568-609f736a93a7 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:40.446294 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f7936e-fa43-49b2-8568-609f736a93a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:40.447317 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f7936e-fa43-49b2-8568-609f736a93a7 None admin] PATH_INFO: `/v3/services` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:40.447317 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f7936e-fa43-49b2-8568-609f736a93a7 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:40.447757 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f7936e-fa43-49b2-8568-609f736a93a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:40.449416 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f7936e-fa43-49b2-8568-609f736a93a7 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:40.459951 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 344/685] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Oct 12 00:02:40 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:40.461740 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f7936e-fa43-49b2-8568-609f736a93a7 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:40.535677 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 342/686] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:02:40 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 00:02:40.579460 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:40.582996 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f7936e-fa43-49b2-8568-609f736a93a7 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:40.618672 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f8e347-57f0-4e19-90f4-c9cdc92d4c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:40.619207 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f8e347-57f0-4e19-90f4-c9cdc92d4c6e None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:40.619516 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f8e347-57f0-4e19-90f4-c9cdc92d4c6e None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:40.619782 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f8e347-57f0-4e19-90f4-c9cdc92d4c6e None admin] PATH_INFO: `/v3/services` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:40.620465 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f8e347-57f0-4e19-90f4-c9cdc92d4c6e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:40.620759 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f8e347-57f0-4e19-90f4-c9cdc92d4c6e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:40.622189 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f8e347-57f0-4e19-90f4-c9cdc92d4c6e None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:40.628255 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 345/687] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Oct 12 00:02:40 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:40.634241 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7487dd09-f9f6-4eda-9e30-36c0e2fdea4c None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:40.738335 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7487dd09-f9f6-4eda-9e30-36c0e2fdea4c None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:40.768151 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:40.773556 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f8f66c-da1a-4378-9804-397a37903ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:40.774225 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f8f66c-da1a-4378-9804-397a37903ab0 None admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:40.774496 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f8f66c-da1a-4378-9804-397a37903ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:40.774763 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f8f66c-da1a-4378-9804-397a37903ab0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:40.775459 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f8f66c-da1a-4378-9804-397a37903ab0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:40.776877 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f8f66c-da1a-4378-9804-397a37903ab0 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:40.840461 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f8e347-57f0-4e19-90f4-c9cdc92d4c6e None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:40.888010 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 343/688] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:02:40 2022] POST /identity/v3/endpoints => generated 347 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:02:40.914148 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965c8e17-7c18-48de-bc91-c92f208402c2 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:40.914495 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965c8e17-7c18-48de-bc91-c92f208402c2 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:40.914697 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965c8e17-7c18-48de-bc91-c92f208402c2 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:40.916380 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 344/689] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:02:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 00:02:40.922028 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb5f78a-d4ba-4fe8-9429-138e8df8b6d0 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:40.922351 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb5f78a-d4ba-4fe8-9429-138e8df8b6d0 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:40.922561 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb5f78a-d4ba-4fe8-9429-138e8df8b6d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:40.942031 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-6fb5f78a-d4ba-4fe8-9429-138e8df8b6d0 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:02:40.943245 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-6fb5f78a-d4ba-4fe8-9429-138e8df8b6d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:02:40.986774 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f8e347-57f0-4e19-90f4-c9cdc92d4c6e None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:41.031740 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 345/690] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:02:40 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 12 00:02:41.036846 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d5767e-e7a5-4d77-94d9-50594a046f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:41.037834 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d5767e-e7a5-4d77-94d9-50594a046f2a None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:41.038341 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d5767e-e7a5-4d77-94d9-50594a046f2a None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:41.038813 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d5767e-e7a5-4d77-94d9-50594a046f2a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:41.051921 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d5767e-e7a5-4d77-94d9-50594a046f2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 00:02:41.057296 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-79d5767e-e7a5-4d77-94d9-50594a046f2a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 12 00:02:41.057296 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:41.057296 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:41.057296 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:41.057296 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:41.057296 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:41.057296 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:41.057296 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:41.057296 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:41.057296 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:41.057296 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:41.057296 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:41.057296 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 00:02:41.057296 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 00:02:41.057296 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 00:02:41.057296 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 00:02:41.058241 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 00:02:41.058241 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 00:02:41.058241 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 00:02:41.058241 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 00:02:41.058241 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:41.058241 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:41.058241 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:41.058241 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:41.058241 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:41.058241 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:41.058241 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:41.058241 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:41.058241 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:41.058241 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:41.058241 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:41.058241 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:41.059083 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:41.059083 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:41.059083 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:41.059083 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:41.059083 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 00:02:41.059083 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 00:02:41.059083 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 00:02:41.059083 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 00:02:41.059083 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 00:02:41.059083 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 00:02:41.059083 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 12 00:02:41.059083 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:41.061893 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 346/691] 10.222.0.36 () {56 vars in 1120 bytes} [Wed Oct 12 00:02:40 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 00:02:41.067583 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb5f78a-d4ba-4fe8-9429-138e8df8b6d0 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:41.171918 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb5f78a-d4ba-4fe8-9429-138e8df8b6d0 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:41.186913 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:41.208307 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f516447b-ea32-4e6b-9d8e-f013942e576a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:41.208929 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f516447b-ea32-4e6b-9d8e-f013942e576a None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:41.209208 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f516447b-ea32-4e6b-9d8e-f013942e576a None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:41.209728 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f516447b-ea32-4e6b-9d8e-f013942e576a None admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:41.210236 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f516447b-ea32-4e6b-9d8e-f013942e576a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:41.210541 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f516447b-ea32-4e6b-9d8e-f013942e576a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:41.211945 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f516447b-ea32-4e6b-9d8e-f013942e576a None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:41.221144 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 346/692] 10.222.0.36 () {56 vars in 1115 bytes} [Wed Oct 12 00:02:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:41.227454 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d5767e-e7a5-4d77-94d9-50594a046f2a None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:41.255116 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:41.298334 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f516447b-ea32-4e6b-9d8e-f013942e576a None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:41.354236 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d5767e-e7a5-4d77-94d9-50594a046f2a None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:41.401152 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b200b8-f8d7-4b5c-a934-fb6a812f37e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:41.402385 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b200b8-f8d7-4b5c-a934-fb6a812f37e2 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:41.402899 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b200b8-f8d7-4b5c-a934-fb6a812f37e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:41.403694 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b200b8-f8d7-4b5c-a934-fb6a812f37e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:41.418276 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b200b8-f8d7-4b5c-a934-fb6a812f37e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:41.420174 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b200b8-f8d7-4b5c-a934-fb6a812f37e2 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:41.420750 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f516447b-ea32-4e6b-9d8e-f013942e576a None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:41.439848 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-d8b200b8-f8d7-4b5c-a934-fb6a812f37e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:41.439848 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:41.439848 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:41.439848 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:41.439848 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:41.439848 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:41.439848 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:41.439848 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:41.439848 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:41.439848 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:41.439848 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:41.439848 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:41.439848 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 00:02:41.439848 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 00:02:41.439848 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 00:02:41.439848 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 00:02:41.440981 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:41.440981 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:41.440981 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:41.440981 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:41.440981 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:41.440981 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:41.440981 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:41.440981 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:41.440981 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:41.440981 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:41.440981 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:41.440981 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:41.440981 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:41.440981 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:41.440981 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:41.440981 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:41.440981 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 00:02:41.442155 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 00:02:41.442155 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:41.442155 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:41.444341 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 347/693] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:02:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:41.450193 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b200b8-f8d7-4b5c-a934-fb6a812f37e2 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:41.465152 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92032eeb-4e9a-43bd-9dea-dd8c140a7d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:41.465882 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92032eeb-4e9a-43bd-9dea-dd8c140a7d07 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:41.466223 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92032eeb-4e9a-43bd-9dea-dd8c140a7d07 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:41.466563 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92032eeb-4e9a-43bd-9dea-dd8c140a7d07 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:41.467579 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92032eeb-4e9a-43bd-9dea-dd8c140a7d07 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:41.470336 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92032eeb-4e9a-43bd-9dea-dd8c140a7d07 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:41.482225 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-92032eeb-4e9a-43bd-9dea-dd8c140a7d07 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 12 00:02:41.482225 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:41.482225 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 00:02:41.482225 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 00:02:41.482225 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 00:02:41.482225 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 00:02:41.482225 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 00:02:41.482225 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 00:02:41.482225 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 12 00:02:41.482225 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:41.482225 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 00:02:41.482225 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:41.482225 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:41.482225 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:41.482225 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:41.482225 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:41.482225 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:41.483258 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:41.483258 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:41.483258 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:41.483258 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:41.483258 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:41.483258 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:41.483258 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 00:02:41.483258 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 00:02:41.483258 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 00:02:41.483258 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 00:02:41.483258 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:41.483258 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:41.483258 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:41.483258 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:41.483258 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:41.483258 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:41.484117 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:41.484117 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:41.484117 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:41.484117 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:41.484117 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:41.484117 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:41.484117 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:41.484117 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:41.484117 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:41.484117 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:41.484117 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 00:02:41.484117 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 00:02:41.484117 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 12 00:02:41.484117 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:41.484117 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 347/694] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Oct 12 00:02:41 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:41.490552 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92032eeb-4e9a-43bd-9dea-dd8c140a7d07 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:41.548094 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:41.549921 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:41.580238 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b200b8-f8d7-4b5c-a934-fb6a812f37e2 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:41.613714 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92032eeb-4e9a-43bd-9dea-dd8c140a7d07 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:41.621952 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f8c66e-4cc2-48ba-92ba-d91ac54419ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:41.622834 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f8c66e-4cc2-48ba-92ba-d91ac54419ca None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:41.623356 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f8c66e-4cc2-48ba-92ba-d91ac54419ca None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:41.623865 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f8c66e-4cc2-48ba-92ba-d91ac54419ca None admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:41.624826 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f8c66e-4cc2-48ba-92ba-d91ac54419ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:41.625319 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f8c66e-4cc2-48ba-92ba-d91ac54419ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:41.627005 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f8c66e-4cc2-48ba-92ba-d91ac54419ca None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:41.640697 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 348/695] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Oct 12 00:02:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:41.655276 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1695968d-553a-4bad-a4df-7fd665623c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:41.655926 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1695968d-553a-4bad-a4df-7fd665623c37 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:41.656205 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1695968d-553a-4bad-a4df-7fd665623c37 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:41.656365 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f8c66e-4cc2-48ba-92ba-d91ac54419ca None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:41.656462 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1695968d-553a-4bad-a4df-7fd665623c37 None admin] PATH_INFO: `/v3/services` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:41.657682 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1695968d-553a-4bad-a4df-7fd665623c37 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:41.657936 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1695968d-553a-4bad-a4df-7fd665623c37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:41.659291 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1695968d-553a-4bad-a4df-7fd665623c37 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:41.664490 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 348/696] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Oct 12 00:02:41 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:41.670787 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1695968d-553a-4bad-a4df-7fd665623c37 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:41.781193 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f8c66e-4cc2-48ba-92ba-d91ac54419ca None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:41.802981 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1695968d-553a-4bad-a4df-7fd665623c37 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:41.805474 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:41.823806 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae589b0c-c939-4e19-9dd7-bb5fa1a80d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:41.824902 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae589b0c-c939-4e19-9dd7-bb5fa1a80d78 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:41.825224 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae589b0c-c939-4e19-9dd7-bb5fa1a80d78 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:41.825556 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae589b0c-c939-4e19-9dd7-bb5fa1a80d78 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:41.829649 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:41.836270 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae589b0c-c939-4e19-9dd7-bb5fa1a80d78 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:41.837761 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae589b0c-c939-4e19-9dd7-bb5fa1a80d78 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:41.841320 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6e47d2-5707-46ad-a5a8-9715833db32b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:41.842162 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6e47d2-5707-46ad-a5a8-9715833db32b None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:41.842672 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6e47d2-5707-46ad-a5a8-9715833db32b None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:41.843142 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6e47d2-5707-46ad-a5a8-9715833db32b None admin] PATH_INFO: `/v3/services` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:41.844010 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6e47d2-5707-46ad-a5a8-9715833db32b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:41.845067 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6e47d2-5707-46ad-a5a8-9715833db32b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:41.850625 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6e47d2-5707-46ad-a5a8-9715833db32b None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:41.851722 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-ae589b0c-c939-4e19-9dd7-bb5fa1a80d78 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 12 00:02:41.851722 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:41.851722 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:41.851722 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:41.851722 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:41.851722 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:41.851722 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:41.851722 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:41.851722 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:41.851722 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:41.851722 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:41.851722 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:41.851722 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 00:02:41.851722 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 00:02:41.851722 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 00:02:41.851722 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 00:02:41.853327 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:41.853327 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:41.853327 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 00:02:41.853327 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 00:02:41.853327 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 00:02:41.853327 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 00:02:41.853327 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:41.853327 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:41.853327 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:41.853327 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:41.853327 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:41.853327 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:41.853327 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:41.853327 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:41.853327 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:41.853327 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:41.853327 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:41.853327 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:41.854490 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:41.854490 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:41.854490 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 00:02:41.854490 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 00:02:41.854490 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 00:02:41.854490 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 00:02:41.854490 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 00:02:41.854490 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 00:02:41.854490 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 12 00:02:41.854490 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:41.855141 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 349/697] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Oct 12 00:02:41 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 00:02:41.856684 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 349/698] 10.222.0.36 () {56 vars in 1117 bytes} [Wed Oct 12 00:02:41 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:41.858135 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6e47d2-5707-46ad-a5a8-9715833db32b None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:41.861788 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae589b0c-c939-4e19-9dd7-bb5fa1a80d78 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:41.972033 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6e47d2-5707-46ad-a5a8-9715833db32b None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:41.987716 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae589b0c-c939-4e19-9dd7-bb5fa1a80d78 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:42.009015 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45454614-ddb0-48c6-8af8-839d97849ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:42.009904 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45454614-ddb0-48c6-8af8-839d97849ea9 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:42.010336 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45454614-ddb0-48c6-8af8-839d97849ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:42.010762 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45454614-ddb0-48c6-8af8-839d97849ea9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:42.011612 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45454614-ddb0-48c6-8af8-839d97849ea9 None admin] RBAC: Adding query filter params (interface=public, service_id=1ee77a2329ba4dc4a2a3459a5df81de4, region_id=RegionOne) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:42.012066 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45454614-ddb0-48c6-8af8-839d97849ea9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:42.013659 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45454614-ddb0-48c6-8af8-839d97849ea9 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:42.020150 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 350/699] 10.222.0.36 () {56 vars in 1251 bytes} [Wed Oct 12 00:02:41 2022] GET /identity/v3/endpoints?service_id=1ee77a2329ba4dc4a2a3459a5df81de4&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:42.031457 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222af1e7-e106-41bf-9513-276902e84b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:42.031623 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45454614-ddb0-48c6-8af8-839d97849ea9 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:42.031721 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222af1e7-e106-41bf-9513-276902e84b1f None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:42.031721 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222af1e7-e106-41bf-9513-276902e84b1f None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:42.031721 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222af1e7-e106-41bf-9513-276902e84b1f None admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:42.031721 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222af1e7-e106-41bf-9513-276902e84b1f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:42.031721 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222af1e7-e106-41bf-9513-276902e84b1f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:42.033321 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222af1e7-e106-41bf-9513-276902e84b1f None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:42.055852 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 350/700] 10.222.0.36 () {56 vars in 1149 bytes} [Wed Oct 12 00:02:41 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 00:02:42.061823 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222af1e7-e106-41bf-9513-276902e84b1f None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:42.063851 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:42.140408 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45454614-ddb0-48c6-8af8-839d97849ea9 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:42.159735 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:42.189175 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222af1e7-e106-41bf-9513-276902e84b1f None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:42.191143 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877a788e-d6d7-4fb3-a965-0aca584be1d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:42.191783 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877a788e-d6d7-4fb3-a965-0aca584be1d3 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:42.192053 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877a788e-d6d7-4fb3-a965-0aca584be1d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:42.192317 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877a788e-d6d7-4fb3-a965-0aca584be1d3 None admin] PATH_INFO: `/v3/services` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:42.193381 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877a788e-d6d7-4fb3-a965-0aca584be1d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:42.194262 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877a788e-d6d7-4fb3-a965-0aca584be1d3 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:42.207829 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 351/701] 10.222.0.36 () {56 vars in 1087 bytes} [Wed Oct 12 00:02:42 2022] GET /identity/v3/services => generated 1497 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:02:42.232101 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90efe74b-a5a0-403a-bdb6-503dfa6a0b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:42.232993 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90efe74b-a5a0-403a-bdb6-503dfa6a0b6d None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:42.233416 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90efe74b-a5a0-403a-bdb6-503dfa6a0b6d None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:42.233828 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90efe74b-a5a0-403a-bdb6-503dfa6a0b6d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:42.315816 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90efe74b-a5a0-403a-bdb6-503dfa6a0b6d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:42.317511 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90efe74b-a5a0-403a-bdb6-503dfa6a0b6d None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:42.378837 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:42.429962 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 351/702] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Oct 12 00:02:42 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 12 00:02:42.435566 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877a788e-d6d7-4fb3-a965-0aca584be1d3 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:42.534764 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:42.541067 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877a788e-d6d7-4fb3-a965-0aca584be1d3 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:42.580334 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95522721-a248-42d3-8788-63794daec251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:42.580991 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95522721-a248-42d3-8788-63794daec251 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:42.581263 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95522721-a248-42d3-8788-63794daec251 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:42.581529 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95522721-a248-42d3-8788-63794daec251 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:42.593085 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95522721-a248-42d3-8788-63794daec251 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:42.594600 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95522721-a248-42d3-8788-63794daec251 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:42.609702 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-95522721-a248-42d3-8788-63794daec251 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:42.609702 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:42.609702 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:42.609702 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:42.609702 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:42.609702 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:42.609702 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:42.609702 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:42.609702 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:42.609702 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:42.609702 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:42.609702 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:42.609702 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 00:02:42.609702 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 00:02:42.609702 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 00:02:42.609702 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 00:02:42.611259 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:42.611259 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:42.611259 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:42.611259 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:42.611259 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:42.611259 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:42.611259 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:42.611259 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:42.611259 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:42.611259 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:42.611259 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:42.611259 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:42.611259 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:42.611259 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:42.611259 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:42.611259 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:42.611259 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 00:02:42.612261 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 00:02:42.612261 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:42.612261 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:42.615724 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 352/703] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:02:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:42.617306 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90efe74b-a5a0-403a-bdb6-503dfa6a0b6d None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:42.684493 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f2219-61f4-4508-bfc8-e9215436f3a6 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:42.684768 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f2219-61f4-4508-bfc8-e9215436f3a6 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:42.685037 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f2219-61f4-4508-bfc8-e9215436f3a6 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:42.686382 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 353/704] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:02:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 00:02:42.691634 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc55712-43a1-4790-9b5c-3df00ee7e6b3 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:42.691911 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc55712-43a1-4790-9b5c-3df00ee7e6b3 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:42.692178 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc55712-43a1-4790-9b5c-3df00ee7e6b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:42.712403 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-ccc55712-43a1-4790-9b5c-3df00ee7e6b3 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:02:42.713591 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-ccc55712-43a1-4790-9b5c-3df00ee7e6b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:02:42.728302 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90efe74b-a5a0-403a-bdb6-503dfa6a0b6d None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:42.764514 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b942f3b1-a2b8-4d15-98ee-720689a71702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:42.765126 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b942f3b1-a2b8-4d15-98ee-720689a71702 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:42.765501 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b942f3b1-a2b8-4d15-98ee-720689a71702 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:42.765773 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b942f3b1-a2b8-4d15-98ee-720689a71702 None admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:42.766931 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b942f3b1-a2b8-4d15-98ee-720689a71702 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:42.767227 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b942f3b1-a2b8-4d15-98ee-720689a71702 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:42.768710 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b942f3b1-a2b8-4d15-98ee-720689a71702 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:42.783387 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 352/705] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Oct 12 00:02:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:42.788534 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b942f3b1-a2b8-4d15-98ee-720689a71702 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:42.792656 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 354/706] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:02:42 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 12 00:02:42.804626 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:42.893526 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b942f3b1-a2b8-4d15-98ee-720689a71702 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:42.927442 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186eb082-c19c-4893-a8f1-c8b9196f0561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:42.928078 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186eb082-c19c-4893-a8f1-c8b9196f0561 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:42.928356 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186eb082-c19c-4893-a8f1-c8b9196f0561 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:42.928628 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186eb082-c19c-4893-a8f1-c8b9196f0561 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:42.935286 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186eb082-c19c-4893-a8f1-c8b9196f0561 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:42.936773 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186eb082-c19c-4893-a8f1-c8b9196f0561 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:42.948868 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-186eb082-c19c-4893-a8f1-c8b9196f0561 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 00:02:42.948868 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:42.948868 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:42.948868 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:42.948868 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:42.948868 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:42.948868 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:42.948868 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:42.948868 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:42.948868 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:42.948868 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:42.948868 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:42.948868 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 00:02:42.948868 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 00:02:42.948868 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 00:02:42.949934 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 00:02:42.949934 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:42.949934 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:42.949934 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:42.949934 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:42.949934 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:42.949934 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:42.949934 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:42.949934 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:42.949934 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:42.949934 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:42.949934 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:42.949934 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:42.949934 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:42.949934 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:42.949934 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:42.949934 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:42.950839 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 00:02:42.950839 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 00:02:42.950839 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 00:02:42.950839 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 00:02:42.950839 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 00:02:42.950839 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 00:02:42.950839 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 00:02:42.950839 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:42.950839 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 353/707] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Oct 12 00:02:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:42.956412 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc55712-43a1-4790-9b5c-3df00ee7e6b3 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:43.037766 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:43.060137 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc55712-43a1-4790-9b5c-3df00ee7e6b3 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:43.082614 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186eb082-c19c-4893-a8f1-c8b9196f0561 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:43.099969 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f624edf7-c694-4bfa-90fb-58bea697bab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:43.100702 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f624edf7-c694-4bfa-90fb-58bea697bab4 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:43.100985 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f624edf7-c694-4bfa-90fb-58bea697bab4 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:43.101254 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f624edf7-c694-4bfa-90fb-58bea697bab4 None admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:43.101966 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f624edf7-c694-4bfa-90fb-58bea697bab4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:43.102268 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f624edf7-c694-4bfa-90fb-58bea697bab4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:43.104002 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f624edf7-c694-4bfa-90fb-58bea697bab4 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:43.122531 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 355/708] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Oct 12 00:02:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:43.123720 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f624edf7-c694-4bfa-90fb-58bea697bab4 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:43.171725 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:43.199207 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186eb082-c19c-4893-a8f1-c8b9196f0561 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:43.233058 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f624edf7-c694-4bfa-90fb-58bea697bab4 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:43.235991 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315d855d-1682-47f8-aea2-c69e77f37964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:43.236421 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315d855d-1682-47f8-aea2-c69e77f37964 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:43.236694 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315d855d-1682-47f8-aea2-c69e77f37964 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:43.236957 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315d855d-1682-47f8-aea2-c69e77f37964 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:43.237526 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315d855d-1682-47f8-aea2-c69e77f37964 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:43.238883 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315d855d-1682-47f8-aea2-c69e77f37964 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:43.248715 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-315d855d-1682-47f8-aea2-c69e77f37964 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 12 00:02:43.248715 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:43.248715 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 00:02:43.248715 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 00:02:43.248715 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 00:02:43.248715 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 00:02:43.248715 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 00:02:43.248715 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 00:02:43.248715 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 12 00:02:43.248715 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:43.248715 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 00:02:43.248715 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:43.248715 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:43.248715 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:43.248715 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:43.248715 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:43.248715 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:43.249726 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:43.249726 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:43.249726 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:43.249726 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:43.249726 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:43.249726 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:43.249726 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 00:02:43.249726 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 00:02:43.249726 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 00:02:43.249726 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 00:02:43.249726 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:43.249726 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:43.249726 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:43.249726 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:43.249726 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:43.249726 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:43.251105 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:43.251105 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:43.251105 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:43.251105 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:43.251105 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:43.251105 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:43.251105 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:43.251105 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:43.251105 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:43.251105 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:43.251105 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 00:02:43.251105 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 00:02:43.251105 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 12 00:02:43.251105 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:43.251105 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 354/709] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Oct 12 00:02:43 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 00:02:43.252744 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f008fe-6572-4c14-a292-533167f4ddc6 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:43.252744 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f008fe-6572-4c14-a292-533167f4ddc6 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:43.252744 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f008fe-6572-4c14-a292-533167f4ddc6 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:43.255495 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 355/710] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:02:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:02:43.256873 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f008fe-6572-4c14-a292-533167f4ddc6 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:43.274023 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab4091c-79bd-4182-8ec2-ddaea032f1a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:43.274629 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab4091c-79bd-4182-8ec2-ddaea032f1a2 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:43.274900 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab4091c-79bd-4182-8ec2-ddaea032f1a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:43.275165 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab4091c-79bd-4182-8ec2-ddaea032f1a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:43.275916 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab4091c-79bd-4182-8ec2-ddaea032f1a2 None admin] RBAC: Adding query filter params (role.id=59b6984c4af34bffa59fdac6d215a6a3, scope.project.id=0697927c6aac43d4b14a750fbbb1145e, user.id=d030d18b595144f19d6ad01d4cd5d52b) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:43.276214 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab4091c-79bd-4182-8ec2-ddaea032f1a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:43.278088 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab4091c-79bd-4182-8ec2-ddaea032f1a2 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:43.284585 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 356/711] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Oct 12 00:02:43 2022] GET /identity/v3/role_assignments?user.id=d030d18b595144f19d6ad01d4cd5d52b&scope.project.id=0697927c6aac43d4b14a750fbbb1145e&role.id=59b6984c4af34bffa59fdac6d215a6a3 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 00:02:43.289349 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1152fee-21a7-4877-98d7-fd59c8de28ad None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:43.289944 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1152fee-21a7-4877-98d7-fd59c8de28ad None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:43.290888 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1152fee-21a7-4877-98d7-fd59c8de28ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:43.299909 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:43.319181 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-d1152fee-21a7-4877-98d7-fd59c8de28ad None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:02:43.321155 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-d1152fee-21a7-4877-98d7-fd59c8de28ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:02:43.373701 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f008fe-6572-4c14-a292-533167f4ddc6 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:43.419423 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 357/712] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:02:43 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 00:02:43.420674 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77685257-0c08-47ba-b62c-8abcf6ad7e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:43.421322 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77685257-0c08-47ba-b62c-8abcf6ad7e31 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:43.421603 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77685257-0c08-47ba-b62c-8abcf6ad7e31 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:43.421928 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77685257-0c08-47ba-b62c-8abcf6ad7e31 None admin] PATH_INFO: `/v3/services` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:43.423062 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77685257-0c08-47ba-b62c-8abcf6ad7e31 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:43.423390 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77685257-0c08-47ba-b62c-8abcf6ad7e31 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:43.424838 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77685257-0c08-47ba-b62c-8abcf6ad7e31 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:43.430694 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 356/713] 10.222.0.36 () {56 vars in 1122 bytes} [Wed Oct 12 00:02:43 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:43.436501 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1152fee-21a7-4877-98d7-fd59c8de28ad None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:43.543564 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1152fee-21a7-4877-98d7-fd59c8de28ad None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:43.575425 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:43.580131 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e361c7ab-61f3-4060-b9e7-ebb88d6ddc06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:43.580747 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e361c7ab-61f3-4060-b9e7-ebb88d6ddc06 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:43.581083 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e361c7ab-61f3-4060-b9e7-ebb88d6ddc06 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:43.581444 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e361c7ab-61f3-4060-b9e7-ebb88d6ddc06 None admin] PATH_INFO: `/v3/services` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:43.582079 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e361c7ab-61f3-4060-b9e7-ebb88d6ddc06 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:43.582384 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e361c7ab-61f3-4060-b9e7-ebb88d6ddc06 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:43.583803 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e361c7ab-61f3-4060-b9e7-ebb88d6ddc06 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:43.588995 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 358/714] 10.222.0.36 () {56 vars in 1122 bytes} [Wed Oct 12 00:02:43 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:43.594415 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77685257-0c08-47ba-b62c-8abcf6ad7e31 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:43.597565 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:43.698322 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77685257-0c08-47ba-b62c-8abcf6ad7e31 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:43.711310 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e361c7ab-61f3-4060-b9e7-ebb88d6ddc06 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:43.733994 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7072503-e1e3-4cc2-ac0b-6fe4e6e45f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:43.734607 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7072503-e1e3-4cc2-ac0b-6fe4e6e45f64 None admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:43.734907 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7072503-e1e3-4cc2-ac0b-6fe4e6e45f64 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:43.735213 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7072503-e1e3-4cc2-ac0b-6fe4e6e45f64 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:43.736009 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7072503-e1e3-4cc2-ac0b-6fe4e6e45f64 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:43.737382 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7072503-e1e3-4cc2-ac0b-6fe4e6e45f64 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:43.803016 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 357/715] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:02:43 2022] POST /identity/v3/endpoints => generated 349 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:02:43.822813 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e361c7ab-61f3-4060-b9e7-ebb88d6ddc06 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:43.875061 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab602168-5791-4ffd-bb66-787e4521cf0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:43.876624 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab602168-5791-4ffd-bb66-787e4521cf0d None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:43.877177 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab602168-5791-4ffd-bb66-787e4521cf0d None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:43.877765 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab602168-5791-4ffd-bb66-787e4521cf0d None admin] PATH_INFO: `/v3/services/compute` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:43.878607 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab602168-5791-4ffd-bb66-787e4521cf0d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:43.880332 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab602168-5791-4ffd-bb66-787e4521cf0d None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:43.891905 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-ab602168-5791-4ffd-bb66-787e4521cf0d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 12 00:02:43.891905 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:43.891905 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 00:02:43.891905 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 00:02:43.891905 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 00:02:43.891905 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 00:02:43.891905 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 00:02:43.891905 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 00:02:43.891905 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 12 00:02:43.891905 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:43.891905 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 00:02:43.891905 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:43.891905 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:43.891905 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:43.891905 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:43.891905 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:43.891905 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:43.892990 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:43.892990 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:43.892990 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:43.892990 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:43.892990 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:43.892990 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:43.892990 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 00:02:43.892990 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 00:02:43.892990 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 00:02:43.892990 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 00:02:43.892990 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:43.892990 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:43.892990 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:43.892990 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:43.892990 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:43.892990 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:43.893815 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:43.893815 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:43.893815 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:43.893815 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:43.893815 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:43.893815 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:43.893815 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:43.893815 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:43.893815 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:43.893815 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:43.893815 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 00:02:43.893815 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 00:02:43.893815 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 12 00:02:43.893815 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:43.896809 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 359/716] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Oct 12 00:02:43 2022] GET /identity/v3/services/compute => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 00:02:43.903503 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7072503-e1e3-4cc2-ac0b-6fe4e6e45f64 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:43.967839 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:44.013534 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7072503-e1e3-4cc2-ac0b-6fe4e6e45f64 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:44.049170 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af31b64-ed81-41c6-ac70-7d67cc97c343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:44.049794 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af31b64-ed81-41c6-ac70-7d67cc97c343 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:44.050103 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af31b64-ed81-41c6-ac70-7d67cc97c343 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:44.050374 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af31b64-ed81-41c6-ac70-7d67cc97c343 None admin] PATH_INFO: `/v3/services` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:44.050999 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af31b64-ed81-41c6-ac70-7d67cc97c343 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:44.051294 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af31b64-ed81-41c6-ac70-7d67cc97c343 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:44.052706 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af31b64-ed81-41c6-ac70-7d67cc97c343 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:44.057870 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 358/717] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Oct 12 00:02:43 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:44.064027 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab602168-5791-4ffd-bb66-787e4521cf0d None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:44.123714 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:44.172783 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab602168-5791-4ffd-bb66-787e4521cf0d None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:44.206240 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d6c379-66b8-477e-b629-bf1382e0b937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:44.207090 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d6c379-66b8-477e-b629-bf1382e0b937 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:44.207537 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d6c379-66b8-477e-b629-bf1382e0b937 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:44.208003 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d6c379-66b8-477e-b629-bf1382e0b937 None admin] PATH_INFO: `/v3/services` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:44.208799 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d6c379-66b8-477e-b629-bf1382e0b937 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:44.209278 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d6c379-66b8-477e-b629-bf1382e0b937 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:44.210779 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d6c379-66b8-477e-b629-bf1382e0b937 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:44.216076 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 360/718] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Oct 12 00:02:44 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:44.491032 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5781c2a2-1ff8-4ed4-bb44-52d08328ebe0 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:44.491480 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5781c2a2-1ff8-4ed4-bb44-52d08328ebe0 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:44.491658 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5781c2a2-1ff8-4ed4-bb44-52d08328ebe0 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:44.493252 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 359/719] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:02:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:02:44.498631 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d219f244-22e9-42a5-833c-92f4e07770c9 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:44.499073 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d219f244-22e9-42a5-833c-92f4e07770c9 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:44.499171 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d219f244-22e9-42a5-833c-92f4e07770c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:44.518821 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-d219f244-22e9-42a5-833c-92f4e07770c9 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:02:44.520042 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-d219f244-22e9-42a5-833c-92f4e07770c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:02:44.600947 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 361/720] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:02:44 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 12 00:02:44.868570 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5781c2a2-1ff8-4ed4-bb44-52d08328ebe0 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:44.982016 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5781c2a2-1ff8-4ed4-bb44-52d08328ebe0 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:45.014692 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:45.018630 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d11198f-e232-4ff2-a400-cfd735791136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:45.019534 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d11198f-e232-4ff2-a400-cfd735791136 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:45.020005 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d11198f-e232-4ff2-a400-cfd735791136 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:45.020487 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d11198f-e232-4ff2-a400-cfd735791136 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:45.021255 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d11198f-e232-4ff2-a400-cfd735791136 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:45.022835 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d11198f-e232-4ff2-a400-cfd735791136 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:45.032024 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-5d11198f-e232-4ff2-a400-cfd735791136 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 12 00:02:45.032024 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:45.032024 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 00:02:45.032024 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 00:02:45.032024 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 00:02:45.032024 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 00:02:45.032024 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 00:02:45.032024 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 00:02:45.032024 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 12 00:02:45.032024 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:45.032024 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 00:02:45.032024 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:45.032024 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:45.032024 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:45.032024 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:45.032024 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:45.032024 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:45.042412 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:45.042412 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:45.042412 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:45.042412 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:45.042412 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:45.042412 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:45.042412 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 00:02:45.042412 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 00:02:45.042412 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 00:02:45.042412 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 00:02:45.042412 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:45.042412 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:45.042412 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:45.042412 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:45.042412 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:45.042412 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:45.043245 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:45.043245 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:45.043245 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:45.043245 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:45.043245 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:45.043245 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:45.043245 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:45.043245 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:45.043245 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:45.043245 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:45.043245 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 00:02:45.043245 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 00:02:45.043245 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 12 00:02:45.043245 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:45.043245 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 360/721] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Oct 12 00:02:44 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:45.044093 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d219f244-22e9-42a5-833c-92f4e07770c9 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:45.100163 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:45.148264 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d219f244-22e9-42a5-833c-92f4e07770c9 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:45.182844 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80467975-862d-4db9-aaf8-73131f76cbbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:45.183850 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80467975-862d-4db9-aaf8-73131f76cbbc None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:45.184324 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80467975-862d-4db9-aaf8-73131f76cbbc None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:45.184789 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80467975-862d-4db9-aaf8-73131f76cbbc None admin] PATH_INFO: `/v3/services` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:45.185622 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80467975-862d-4db9-aaf8-73131f76cbbc None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:45.186135 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80467975-862d-4db9-aaf8-73131f76cbbc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:45.187715 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80467975-862d-4db9-aaf8-73131f76cbbc None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:45.193746 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 362/722] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Oct 12 00:02:45 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:45.200446 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d11198f-e232-4ff2-a400-cfd735791136 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:45.306391 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d11198f-e232-4ff2-a400-cfd735791136 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:45.341067 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be7862f-93f1-4b47-96ce-7fba3a6cf59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:45.341674 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be7862f-93f1-4b47-96ce-7fba3a6cf59f None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:45.341950 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be7862f-93f1-4b47-96ce-7fba3a6cf59f None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:45.342216 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be7862f-93f1-4b47-96ce-7fba3a6cf59f None admin] PATH_INFO: `/v3/services` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:45.342838 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be7862f-93f1-4b47-96ce-7fba3a6cf59f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:45.343132 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be7862f-93f1-4b47-96ce-7fba3a6cf59f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:45.345063 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be7862f-93f1-4b47-96ce-7fba3a6cf59f None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:45.350255 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 361/723] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Oct 12 00:02:45 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:45.356061 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80467975-862d-4db9-aaf8-73131f76cbbc None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:45.466863 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80467975-862d-4db9-aaf8-73131f76cbbc None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:45.486854 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:45.507071 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b128da-7084-4fae-820b-3287fc7b03e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:45.508715 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b128da-7084-4fae-820b-3287fc7b03e0 None admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:45.509227 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b128da-7084-4fae-820b-3287fc7b03e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:45.509700 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b128da-7084-4fae-820b-3287fc7b03e0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:45.510578 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b128da-7084-4fae-820b-3287fc7b03e0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:45.512170 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b128da-7084-4fae-820b-3287fc7b03e0 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:45.561639 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cc0002-6381-487e-9d78-2e163667325e None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:45.561928 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cc0002-6381-487e-9d78-2e163667325e None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:45.562204 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cc0002-6381-487e-9d78-2e163667325e None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:45.564241 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 362/724] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:02:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 00:02:45.568892 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e7a823-5bd7-4f86-8304-9c55225ae2c6 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:45.569180 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e7a823-5bd7-4f86-8304-9c55225ae2c6 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:45.569459 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e7a823-5bd7-4f86-8304-9c55225ae2c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:45.588012 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-04e7a823-5bd7-4f86-8304-9c55225ae2c6 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:02:45.589219 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-04e7a823-5bd7-4f86-8304-9c55225ae2c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:02:45.614659 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 363/725] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:02:45 2022] POST /identity/v3/endpoints => generated 346 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:02:45.644384 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:45.675100 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 363/726] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:02:45 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 12 00:02:45.946515 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b128da-7084-4fae-820b-3287fc7b03e0 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:46.057885 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b128da-7084-4fae-820b-3287fc7b03e0 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:46.095572 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a4a387-b38e-4c60-a287-06e0e3073d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:46.096452 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a4a387-b38e-4c60-a287-06e0e3073d0f None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:46.096870 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a4a387-b38e-4c60-a287-06e0e3073d0f None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:46.097281 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a4a387-b38e-4c60-a287-06e0e3073d0f None admin] PATH_INFO: `/v3/services/image` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:46.098024 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a4a387-b38e-4c60-a287-06e0e3073d0f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:46.099535 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a4a387-b38e-4c60-a287-06e0e3073d0f None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:46.108767 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-f0a4a387-b38e-4c60-a287-06e0e3073d0f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 12 00:02:46.108767 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:46.108767 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 00:02:46.108767 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 00:02:46.108767 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 00:02:46.108767 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 00:02:46.108767 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 00:02:46.108767 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 00:02:46.108767 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 12 00:02:46.108767 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:46.108767 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 00:02:46.108767 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:46.108767 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:46.108767 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:46.108767 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:46.108767 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:46.108767 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:46.110048 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:46.110048 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:46.110048 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:46.110048 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:46.110048 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:46.110048 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:46.110048 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 00:02:46.110048 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 00:02:46.110048 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 00:02:46.110048 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 00:02:46.110048 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:46.110048 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:46.110048 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:46.110048 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:46.110048 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:46.110048 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:46.111046 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:46.111046 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:46.111046 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:46.111046 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:46.111046 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:46.111046 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:46.111046 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:46.111046 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:46.111046 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:46.111046 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:46.111046 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 00:02:46.111046 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 00:02:46.111046 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 12 00:02:46.111046 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:46.113432 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 364/727] 10.222.0.36 () {56 vars in 1105 bytes} [Wed Oct 12 00:02:45 2022] GET /identity/v3/services/image => generated 86 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:46.119438 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e7a823-5bd7-4f86-8304-9c55225ae2c6 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:46.157005 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef0000-d776-4209-a6b4-ec41a1db3526 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:46.157439 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef0000-d776-4209-a6b4-ec41a1db3526 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:46.157814 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef0000-d776-4209-a6b4-ec41a1db3526 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:46.159529 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 365/728] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:02:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:02:46.166224 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a603556-1446-4b53-850f-c2043bd993ab None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:46.166568 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a603556-1446-4b53-850f-c2043bd993ab None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:46.166849 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a603556-1446-4b53-850f-c2043bd993ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:46.187219 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-0a603556-1446-4b53-850f-c2043bd993ab None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:02:46.189261 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-0a603556-1446-4b53-850f-c2043bd993ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:02:46.227271 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:46.229924 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e7a823-5bd7-4f86-8304-9c55225ae2c6 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:46.268824 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a606bf-5509-4345-a0bd-3b4606f36087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:46.269588 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a606bf-5509-4345-a0bd-3b4606f36087 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:46.269962 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a606bf-5509-4345-a0bd-3b4606f36087 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:46.270332 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a606bf-5509-4345-a0bd-3b4606f36087 None admin] PATH_INFO: `/v3/services` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:46.271064 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a606bf-5509-4345-a0bd-3b4606f36087 None admin] RBAC: Adding query filter params (name=image) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:46.271184 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 366/729] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:02:46 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 00:02:46.274445 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a606bf-5509-4345-a0bd-3b4606f36087 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:46.275912 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a606bf-5509-4345-a0bd-3b4606f36087 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:46.281391 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 364/730] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:02:46 2022] GET /identity/v3/services?name=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:46.291634 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a603556-1446-4b53-850f-c2043bd993ab None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:46.403662 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a603556-1446-4b53-850f-c2043bd993ab None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:46.439757 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bf9121-99ed-4db5-8fce-de95d6f12fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:46.440360 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bf9121-99ed-4db5-8fce-de95d6f12fc7 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:46.440633 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bf9121-99ed-4db5-8fce-de95d6f12fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:46.440895 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bf9121-99ed-4db5-8fce-de95d6f12fc7 None admin] PATH_INFO: `/v3/services` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:46.441509 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bf9121-99ed-4db5-8fce-de95d6f12fc7 None admin] RBAC: Adding query filter params (type=image) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:46.441802 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bf9121-99ed-4db5-8fce-de95d6f12fc7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:46.443149 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bf9121-99ed-4db5-8fce-de95d6f12fc7 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:46.448040 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 367/731] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:02:46 2022] GET /identity/v3/services?type=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:46.498610 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ade752b-84a4-4622-9e0e-e64a9a1b4312 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:46.498784 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ade752b-84a4-4622-9e0e-e64a9a1b4312 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:46.499064 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ade752b-84a4-4622-9e0e-e64a9a1b4312 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:46.501291 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 365/732] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:02:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:02:46.506899 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ff6a7b-e3fb-4883-8dc0-564735c61f2b None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:46.507193 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ff6a7b-e3fb-4883-8dc0-564735c61f2b None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:46.507756 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ff6a7b-e3fb-4883-8dc0-564735c61f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:46.527752 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-22ff6a7b-e3fb-4883-8dc0-564735c61f2b None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:02:46.528969 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-22ff6a7b-e3fb-4883-8dc0-564735c61f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:02:46.550934 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ade752b-84a4-4622-9e0e-e64a9a1b4312 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:46.571166 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:46.601159 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:46.632744 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 368/733] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:02:46 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 00:02:46.669300 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ade752b-84a4-4622-9e0e-e64a9a1b4312 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:46.704603 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323c09a0-a150-44d9-812d-797004bbbfac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:46.705239 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323c09a0-a150-44d9-812d-797004bbbfac None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:46.705514 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323c09a0-a150-44d9-812d-797004bbbfac None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:46.705784 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323c09a0-a150-44d9-812d-797004bbbfac None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:46.706365 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323c09a0-a150-44d9-812d-797004bbbfac None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:46.707754 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323c09a0-a150-44d9-812d-797004bbbfac None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:46.716591 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-323c09a0-a150-44d9-812d-797004bbbfac None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 12 00:02:46.716591 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:46.716591 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 00:02:46.716591 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 00:02:46.716591 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 00:02:46.716591 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 00:02:46.716591 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 00:02:46.716591 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 00:02:46.716591 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 12 00:02:46.716591 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:46.716591 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 00:02:46.716591 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:46.716591 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:46.716591 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:46.716591 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:46.716591 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:46.716591 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:46.717564 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:46.717564 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:46.717564 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:46.717564 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:46.717564 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:46.717564 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:46.717564 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 00:02:46.717564 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 00:02:46.717564 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 00:02:46.717564 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 00:02:46.717564 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:46.717564 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:46.717564 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:46.717564 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:46.717564 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:46.717564 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:46.718446 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:46.718446 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:46.718446 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:46.718446 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:46.718446 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:46.718446 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:46.718446 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:46.718446 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:46.718446 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:46.718446 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:46.718446 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 00:02:46.718446 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 00:02:46.718446 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 12 00:02:46.718446 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:46.718446 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 366/734] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Oct 12 00:02:46 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:46.724459 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ff6a7b-e3fb-4883-8dc0-564735c61f2b None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:46.833149 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ff6a7b-e3fb-4883-8dc0-564735c61f2b None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:46.868083 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdc45bf-1a0f-4025-99be-b4508d69ef9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:46.868767 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc45bf-1a0f-4025-99be-b4508d69ef9a None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:46.869046 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc45bf-1a0f-4025-99be-b4508d69ef9a None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:46.869314 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc45bf-1a0f-4025-99be-b4508d69ef9a None admin] PATH_INFO: `/v3/services` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:46.869945 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdc45bf-1a0f-4025-99be-b4508d69ef9a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:46.870251 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdc45bf-1a0f-4025-99be-b4508d69ef9a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:46.871702 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdc45bf-1a0f-4025-99be-b4508d69ef9a None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:46.877026 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 369/735] 10.222.0.36 () {56 vars in 1122 bytes} [Wed Oct 12 00:02:46 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:46.882630 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323c09a0-a150-44d9-812d-797004bbbfac None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:46.923503 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdc45bf-1a0f-4025-99be-b4508d69ef9a None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:46.954046 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:46.993319 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323c09a0-a150-44d9-812d-797004bbbfac None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:47.019830 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:47.032328 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540259ba-1dab-4dde-a457-1c41bca53511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:47.033303 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540259ba-1dab-4dde-a457-1c41bca53511 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:47.033781 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540259ba-1dab-4dde-a457-1c41bca53511 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:47.034352 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540259ba-1dab-4dde-a457-1c41bca53511 None admin] PATH_INFO: `/v3/services` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:47.035174 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540259ba-1dab-4dde-a457-1c41bca53511 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:47.039979 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540259ba-1dab-4dde-a457-1c41bca53511 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:47.039979 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540259ba-1dab-4dde-a457-1c41bca53511 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:47.048030 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdc45bf-1a0f-4025-99be-b4508d69ef9a None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:47.048297 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 367/736] 10.222.0.36 () {56 vars in 1122 bytes} [Wed Oct 12 00:02:46 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:47.050255 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540259ba-1dab-4dde-a457-1c41bca53511 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:47.091436 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97241f7f-c7c5-4a89-ad3e-0309ef6f1e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:47.092477 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97241f7f-c7c5-4a89-ad3e-0309ef6f1e7b None admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:47.092838 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97241f7f-c7c5-4a89-ad3e-0309ef6f1e7b None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:47.093188 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97241f7f-c7c5-4a89-ad3e-0309ef6f1e7b None admin] PATH_INFO: `/v3/services` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:47.093926 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97241f7f-c7c5-4a89-ad3e-0309ef6f1e7b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:47.096412 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97241f7f-c7c5-4a89-ad3e-0309ef6f1e7b None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:47.161363 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540259ba-1dab-4dde-a457-1c41bca53511 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:47.164199 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 370/737] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Oct 12 00:02:46 2022] POST /identity/v3/services => generated 242 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 00:02:47.216206 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0769317d-a8f5-440c-9330-1d136a4b4a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:47.216206 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0769317d-a8f5-440c-9330-1d136a4b4a3b None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:47.216206 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0769317d-a8f5-440c-9330-1d136a4b4a3b None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:47.216206 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0769317d-a8f5-440c-9330-1d136a4b4a3b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:47.216638 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0769317d-a8f5-440c-9330-1d136a4b4a3b None admin] RBAC: Adding query filter params (interface=admin, service_id=133212c5ae9942d68601e37894bc1428, region_id=RegionOne) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:47.216638 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0769317d-a8f5-440c-9330-1d136a4b4a3b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:47.216638 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0769317d-a8f5-440c-9330-1d136a4b4a3b None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:47.227097 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 368/738] 10.222.0.36 () {56 vars in 1249 bytes} [Wed Oct 12 00:02:47 2022] GET /identity/v3/endpoints?service_id=133212c5ae9942d68601e37894bc1428&interface=admin®ion_id=RegionOne => generated 193 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 00:02:47.233177 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97241f7f-c7c5-4a89-ad3e-0309ef6f1e7b None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:47.329316 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:47.350316 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97241f7f-c7c5-4a89-ad3e-0309ef6f1e7b None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:47.386476 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3f8027-0d52-4350-9106-29f6b5285024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:47.387093 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3f8027-0d52-4350-9106-29f6b5285024 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:47.387348 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3f8027-0d52-4350-9106-29f6b5285024 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:47.387669 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3f8027-0d52-4350-9106-29f6b5285024 None admin] PATH_INFO: `/v3/services` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:47.388289 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3f8027-0d52-4350-9106-29f6b5285024 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:47.389649 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3f8027-0d52-4350-9106-29f6b5285024 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:47.397012 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 371/739] 10.222.0.36 () {56 vars in 1087 bytes} [Wed Oct 12 00:02:47 2022] GET /identity/v3/services => generated 1727 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:02:47.865577 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644708be-bd04-4cd5-b011-79833856f590 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:47.865966 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644708be-bd04-4cd5-b011-79833856f590 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:47.866150 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644708be-bd04-4cd5-b011-79833856f590 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:47.867797 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 369/740] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:02:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:02:47.873145 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47422188-c730-4904-88cf-c3c8bacb49ae None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:47.873603 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47422188-c730-4904-88cf-c3c8bacb49ae None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:47.873680 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47422188-c730-4904-88cf-c3c8bacb49ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:47.892849 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-47422188-c730-4904-88cf-c3c8bacb49ae None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:02:47.894047 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-47422188-c730-4904-88cf-c3c8bacb49ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:02:47.967464 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 372/741] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:02:47 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 00:02:48.237019 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644708be-bd04-4cd5-b011-79833856f590 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:48.267916 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:48.348391 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644708be-bd04-4cd5-b011-79833856f590 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:48.386142 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a88ac5-a5d9-47ef-9ee7-ae68c71c35ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:48.386754 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a88ac5-a5d9-47ef-9ee7-ae68c71c35ee None admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:48.387030 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a88ac5-a5d9-47ef-9ee7-ae68c71c35ee None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:48.387301 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a88ac5-a5d9-47ef-9ee7-ae68c71c35ee None admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:48.387991 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a88ac5-a5d9-47ef-9ee7-ae68c71c35ee None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:48.389423 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a88ac5-a5d9-47ef-9ee7-ae68c71c35ee None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:48.392707 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: WARNING keystone.server.flask.common [None req-70a88ac5-a5d9-47ef-9ee7-ae68c71c35ee None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 12 00:02:48.447929 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.sql.core [None req-70a88ac5-a5d9-47ef-9ee7-ae68c71c35ee None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 12 00:02:48.447929 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 12 00:02:48.447929 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [parameters: {'id': '3fc571ca446943e6858c0eccf2ef64e1', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 12 00:02:48.447929 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81746) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 12 00:02:48.466213 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-70a88ac5-a5d9-47ef-9ee7-ae68c71c35ee None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 3fc571ca446943e6858c0eccf2ef64e1.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 3fc571ca446943e6858c0eccf2ef64e1. Oct 12 00:02:48.466213 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:48.466213 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 12 00:02:48.466213 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 12 00:02:48.466213 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 12 00:02:48.466213 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 12 00:02:48.466213 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 12 00:02:48.466213 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application result = self._query(query) Oct 12 00:02:48.466213 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 12 00:02:48.466213 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application conn.query(q) Oct 12 00:02:48.466213 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 12 00:02:48.466213 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 12 00:02:48.466213 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 12 00:02:48.467175 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application result.read() Oct 12 00:02:48.467175 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 12 00:02:48.467175 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 12 00:02:48.467175 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 12 00:02:48.467175 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 12 00:02:48.467175 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 12 00:02:48.467175 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 12 00:02:48.467175 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 12 00:02:48.467175 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 12 00:02:48.467175 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 12 00:02:48.467175 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:48.467175 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 12 00:02:48.467175 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:48.467175 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:48.467175 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 12 00:02:48.467175 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 12 00:02:48.467175 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 12 00:02:48.467175 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 12 00:02:48.468174 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 12 00:02:48.468174 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application next(self.gen) Oct 12 00:02:48.468174 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 12 00:02:48.468174 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application yield resource Oct 12 00:02:48.468174 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 12 00:02:48.468174 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application next(self.gen) Oct 12 00:02:48.468174 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 12 00:02:48.468174 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self.session.rollback() Oct 12 00:02:48.468174 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 12 00:02:48.468174 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self.force_reraise() Oct 12 00:02:48.468174 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 12 00:02:48.468174 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise self.value Oct 12 00:02:48.468174 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 12 00:02:48.468174 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 12 00:02:48.468174 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 12 00:02:48.468174 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application session.commit() Oct 12 00:02:48.468174 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 12 00:02:48.469111 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 12 00:02:48.469111 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 12 00:02:48.469111 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self._prepare_impl() Oct 12 00:02:48.469111 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 12 00:02:48.469111 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self.session.flush() Oct 12 00:02:48.469111 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 12 00:02:48.469111 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self._flush(objects) Oct 12 00:02:48.469111 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 12 00:02:48.469111 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 12 00:02:48.469111 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 12 00:02:48.469111 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application compat.raise_( Oct 12 00:02:48.469111 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 12 00:02:48.469111 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception Oct 12 00:02:48.469111 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 12 00:02:48.469111 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application flush_context.execute() Oct 12 00:02:48.469111 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 12 00:02:48.469111 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rec.execute(self) Oct 12 00:02:48.470029 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 12 00:02:48.470029 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 12 00:02:48.470029 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 12 00:02:48.470029 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 12 00:02:48.470029 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 12 00:02:48.470029 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 12 00:02:48.470029 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 12 00:02:48.470029 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 12 00:02:48.470029 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 12 00:02:48.470029 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 12 00:02:48.470029 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 12 00:02:48.470029 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 12 00:02:48.470029 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 12 00:02:48.470029 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 12 00:02:48.470029 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 12 00:02:48.470824 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 12 00:02:48.470824 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 12 00:02:48.470824 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception Oct 12 00:02:48.470824 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 12 00:02:48.470824 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 12 00:02:48.470824 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 12 00:02:48.470824 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 12 00:02:48.470824 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 12 00:02:48.470824 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application result = self._query(query) Oct 12 00:02:48.470824 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 12 00:02:48.470824 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application conn.query(q) Oct 12 00:02:48.470824 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 12 00:02:48.470824 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 12 00:02:48.470824 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 12 00:02:48.470824 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application result.read() Oct 12 00:02:48.470824 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 12 00:02:48.470824 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 12 00:02:48.472983 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 12 00:02:48.472983 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 12 00:02:48.472983 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 12 00:02:48.472983 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 12 00:02:48.472983 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 12 00:02:48.472983 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 12 00:02:48.472983 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 12 00:02:48.472983 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 12 00:02:48.472983 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [parameters: {'id': '3fc571ca446943e6858c0eccf2ef64e1', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 12 00:02:48.472983 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 12 00:02:48.472983 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:48.472983 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 00:02:48.472983 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:48.472983 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:48.472983 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 12 00:02:48.473972 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 12 00:02:48.473972 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 12 00:02:48.473972 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 12 00:02:48.473972 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Oct 12 00:02:48.473972 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:48.473972 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 00:02:48.473972 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:48.473972 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:48.473972 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:48.473972 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:48.473972 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:48.473972 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:48.473972 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:48.473972 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:48.473972 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:48.473972 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:48.473972 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:48.475087 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:48.475087 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Oct 12 00:02:48.475087 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Oct 12 00:02:48.475087 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:48.475087 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:48.475087 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 12 00:02:48.475087 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 12 00:02:48.475087 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 3fc571ca446943e6858c0eccf2ef64e1. Oct 12 00:02:48.475087 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:48.475087 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 370/742] 10.222.0.36 () {60 vars in 1140 bytes} [Wed Oct 12 00:02:48 2022] POST /identity/v3/projects => generated 262 bytes in 231 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:48.479414 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47422188-c730-4904-88cf-c3c8bacb49ae None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:48.559740 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:48.580071 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47422188-c730-4904-88cf-c3c8bacb49ae None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:48.615144 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21edf853-7bfb-4e8f-ae77-7b24dbb91d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:48.615805 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21edf853-7bfb-4e8f-ae77-7b24dbb91d55 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:48.616308 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21edf853-7bfb-4e8f-ae77-7b24dbb91d55 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:48.616576 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21edf853-7bfb-4e8f-ae77-7b24dbb91d55 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:48.623500 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21edf853-7bfb-4e8f-ae77-7b24dbb91d55 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:48.624978 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21edf853-7bfb-4e8f-ae77-7b24dbb91d55 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:48.638550 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-21edf853-7bfb-4e8f-ae77-7b24dbb91d55 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 00:02:48.638550 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:48.638550 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:48.638550 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:48.638550 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:48.638550 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:48.638550 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:48.638550 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:48.638550 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:48.638550 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:48.638550 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:48.638550 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:48.638550 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 00:02:48.638550 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 00:02:48.638550 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 00:02:48.640598 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 00:02:48.640598 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:48.640598 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:48.640598 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:48.640598 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:48.640598 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:48.640598 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:48.640598 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:48.640598 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:48.640598 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:48.640598 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:48.640598 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:48.640598 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:48.640598 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:48.640598 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:48.640598 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:48.640598 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:48.641918 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 00:02:48.641918 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 00:02:48.641918 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 00:02:48.641918 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 00:02:48.641918 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 00:02:48.641918 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 00:02:48.641918 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 00:02:48.641918 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:48.641918 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 373/743] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Oct 12 00:02:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:48.648909 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a88ac5-a5d9-47ef-9ee7-ae68c71c35ee None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:48.743340 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:48.753824 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a02f66-e133-4a01-a820-234023f52073 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:48.759749 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a02f66-e133-4a01-a820-234023f52073 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:48.759749 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a02f66-e133-4a01-a820-234023f52073 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:48.759749 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 374/744] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:02:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 00:02:48.760249 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a88ac5-a5d9-47ef-9ee7-ae68c71c35ee None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:48.760953 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d7e134-f46c-47db-9da8-35c968800bf7 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:48.761241 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d7e134-f46c-47db-9da8-35c968800bf7 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:48.761516 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d7e134-f46c-47db-9da8-35c968800bf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:48.779415 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-90d7e134-f46c-47db-9da8-35c968800bf7 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:02:48.780616 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-90d7e134-f46c-47db-9da8-35c968800bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:02:48.801446 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137a791a-e743-4799-932b-5c65da5e7d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:48.802559 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137a791a-e743-4799-932b-5c65da5e7d92 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:48.803437 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137a791a-e743-4799-932b-5c65da5e7d92 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:48.803894 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137a791a-e743-4799-932b-5c65da5e7d92 None admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:48.804734 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137a791a-e743-4799-932b-5c65da5e7d92 None admin] RBAC: Adding query filter params (name=service) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:48.805288 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137a791a-e743-4799-932b-5c65da5e7d92 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:48.808044 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137a791a-e743-4799-932b-5c65da5e7d92 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:48.836959 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 371/745] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Oct 12 00:02:48 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:48.860732 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 375/746] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:02:48 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 00:02:49.128375 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137a791a-e743-4799-932b-5c65da5e7d92 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:49.231811 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137a791a-e743-4799-932b-5c65da5e7d92 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:49.385659 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fc31a4-9bac-4dba-87ea-c909b62d0e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:49.386282 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fc31a4-9bac-4dba-87ea-c909b62d0e91 None admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:49.386469 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fc31a4-9bac-4dba-87ea-c909b62d0e91 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:49.386741 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fc31a4-9bac-4dba-87ea-c909b62d0e91 None admin] PATH_INFO: `/v3/services` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:49.387423 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fc31a4-9bac-4dba-87ea-c909b62d0e91 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:49.388843 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fc31a4-9bac-4dba-87ea-c909b62d0e91 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:49.469901 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 372/747] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Oct 12 00:02:49 2022] POST /identity/v3/services => generated 242 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:02:49.493576 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b9832-9106-4f3a-b98b-c1b5fde536bc None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:49.493576 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b9832-9106-4f3a-b98b-c1b5fde536bc None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:49.493576 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b9832-9106-4f3a-b98b-c1b5fde536bc None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:49.494555 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 376/748] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:02:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:02:49.501367 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b816e7e6-cd12-4af4-9f0b-098e727fb1b6 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:49.501659 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b816e7e6-cd12-4af4-9f0b-098e727fb1b6 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:49.501967 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b816e7e6-cd12-4af4-9f0b-098e727fb1b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:49.509936 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:49.522874 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-b816e7e6-cd12-4af4-9f0b-098e727fb1b6 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:02:49.524597 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-b816e7e6-cd12-4af4-9f0b-098e727fb1b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:02:49.623236 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 373/749] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:02:49 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 00:02:49.685360 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50a3ecb-8f5f-4504-b021-58ec40adeabd None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:49.685913 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50a3ecb-8f5f-4504-b021-58ec40adeabd None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:49.686388 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50a3ecb-8f5f-4504-b021-58ec40adeabd None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:49.688853 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 377/750] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:02:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:02:49.694440 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d7c8b9-16a4-4002-8e72-9f6f1faddba8 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:49.694990 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d7c8b9-16a4-4002-8e72-9f6f1faddba8 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:49.695508 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d7c8b9-16a4-4002-8e72-9f6f1faddba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:49.714524 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-38d7c8b9-16a4-4002-8e72-9f6f1faddba8 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:02:49.715734 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-38d7c8b9-16a4-4002-8e72-9f6f1faddba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:02:49.786466 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 374/751] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:02:49 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 00:02:49.902564 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50a3ecb-8f5f-4504-b021-58ec40adeabd None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:49.971252 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:50.007271 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50a3ecb-8f5f-4504-b021-58ec40adeabd None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:50.044643 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2976d6da-e15e-4819-a059-bc0dca80eeeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:50.045277 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2976d6da-e15e-4819-a059-bc0dca80eeeb None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:50.045549 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2976d6da-e15e-4819-a059-bc0dca80eeeb None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:50.045816 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2976d6da-e15e-4819-a059-bc0dca80eeeb None admin] PATH_INFO: `/v3/services/compute` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:50.046388 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2976d6da-e15e-4819-a059-bc0dca80eeeb None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:50.048450 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2976d6da-e15e-4819-a059-bc0dca80eeeb None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:50.057448 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-2976d6da-e15e-4819-a059-bc0dca80eeeb None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 12 00:02:50.057448 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:50.057448 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 00:02:50.057448 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 00:02:50.057448 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 00:02:50.057448 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 00:02:50.057448 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 00:02:50.057448 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 00:02:50.057448 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 12 00:02:50.057448 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:50.057448 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 00:02:50.057448 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:50.057448 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:50.057448 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:50.057448 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:50.057448 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:50.057448 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:50.058512 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:50.058512 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:50.058512 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:50.058512 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:50.058512 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:50.058512 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:50.058512 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 00:02:50.058512 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 00:02:50.058512 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 00:02:50.058512 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 00:02:50.058512 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:50.058512 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:50.058512 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:50.058512 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:50.058512 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:50.058512 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:50.059341 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:50.059341 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:50.059341 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:50.059341 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:50.059341 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:50.059341 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:50.059341 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:50.059341 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:50.059341 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:50.059341 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:50.059341 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 00:02:50.059341 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 00:02:50.059341 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 12 00:02:50.059341 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:50.059341 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 378/752] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Oct 12 00:02:49 2022] GET /identity/v3/services/compute => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:50.064543 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d7c8b9-16a4-4002-8e72-9f6f1faddba8 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:50.088727 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2976d6da-e15e-4819-a059-bc0dca80eeeb None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:50.128059 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:50.173878 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d7c8b9-16a4-4002-8e72-9f6f1faddba8 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:50.198168 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2976d6da-e15e-4819-a059-bc0dca80eeeb None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:50.211720 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b294168-fc58-4276-9f4c-a30c53769726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:50.212404 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b294168-fc58-4276-9f4c-a30c53769726 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:50.212681 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b294168-fc58-4276-9f4c-a30c53769726 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:50.212952 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b294168-fc58-4276-9f4c-a30c53769726 None admin] PATH_INFO: `/v3/services` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:50.220115 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b294168-fc58-4276-9f4c-a30c53769726 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:50.220115 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b294168-fc58-4276-9f4c-a30c53769726 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:50.220115 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b294168-fc58-4276-9f4c-a30c53769726 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:50.222183 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 375/753] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Oct 12 00:02:50 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:50.227468 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b294168-fc58-4276-9f4c-a30c53769726 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:50.233758 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:50.240211 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17adb26e-bbb3-479b-8640-9b33c01561f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:50.241093 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17adb26e-bbb3-479b-8640-9b33c01561f2 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:50.241564 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17adb26e-bbb3-479b-8640-9b33c01561f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:50.243552 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17adb26e-bbb3-479b-8640-9b33c01561f2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:50.243552 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17adb26e-bbb3-479b-8640-9b33c01561f2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:50.244713 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17adb26e-bbb3-479b-8640-9b33c01561f2 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:50.254787 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-17adb26e-bbb3-479b-8640-9b33c01561f2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 12 00:02:50.254787 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:50.254787 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 00:02:50.254787 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 00:02:50.254787 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 00:02:50.254787 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 00:02:50.254787 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 00:02:50.254787 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 00:02:50.254787 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 12 00:02:50.254787 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:50.254787 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 00:02:50.254787 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:50.254787 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:50.254787 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:50.254787 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:50.254787 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:50.254787 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:50.256036 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:50.256036 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:50.256036 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:50.256036 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:50.256036 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:50.256036 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:50.256036 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 00:02:50.256036 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 00:02:50.256036 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 00:02:50.256036 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 00:02:50.256036 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:50.256036 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:50.256036 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:50.256036 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:50.256036 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:50.256036 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:50.257063 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:50.257063 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:50.257063 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:50.257063 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:50.257063 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:50.257063 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:50.257063 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:50.257063 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:50.257063 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:50.257063 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:50.257063 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 00:02:50.257063 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 00:02:50.257063 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 12 00:02:50.257063 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:50.265091 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 379/754] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Oct 12 00:02:50 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 00:02:50.266838 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17adb26e-bbb3-479b-8640-9b33c01561f2 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:50.345106 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b294168-fc58-4276-9f4c-a30c53769726 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:50.375504 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:50.386972 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ec94d1-dfc7-4349-a2ec-0b4fe47fde4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:50.387743 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ec94d1-dfc7-4349-a2ec-0b4fe47fde4e None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:50.387970 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ec94d1-dfc7-4349-a2ec-0b4fe47fde4e None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:50.388148 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ec94d1-dfc7-4349-a2ec-0b4fe47fde4e None admin] PATH_INFO: `/v3/services` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:50.388786 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ec94d1-dfc7-4349-a2ec-0b4fe47fde4e None admin] RBAC: Adding query filter params (type=compute) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:50.389088 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ec94d1-dfc7-4349-a2ec-0b4fe47fde4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:50.390531 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ec94d1-dfc7-4349-a2ec-0b4fe47fde4e None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:50.396736 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 376/755] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Oct 12 00:02:50 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:50.403753 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ec94d1-dfc7-4349-a2ec-0b4fe47fde4e None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:50.506865 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17adb26e-bbb3-479b-8640-9b33c01561f2 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:50.535400 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ec94d1-dfc7-4349-a2ec-0b4fe47fde4e None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:50.551285 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72636f84-ab99-4757-afec-2264d7919079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:50.552306 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72636f84-ab99-4757-afec-2264d7919079 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:50.552790 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72636f84-ab99-4757-afec-2264d7919079 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:50.553256 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72636f84-ab99-4757-afec-2264d7919079 None admin] PATH_INFO: `/v3/services` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:50.554091 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72636f84-ab99-4757-afec-2264d7919079 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:50.554811 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72636f84-ab99-4757-afec-2264d7919079 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:50.556797 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72636f84-ab99-4757-afec-2264d7919079 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:50.568081 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 380/756] 10.222.0.36 () {56 vars in 1122 bytes} [Wed Oct 12 00:02:50 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:50.570048 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72636f84-ab99-4757-afec-2264d7919079 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:50.577374 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21b33f5-14b0-45e4-951b-6b1dfb56c155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:50.578045 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21b33f5-14b0-45e4-951b-6b1dfb56c155 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:50.578939 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21b33f5-14b0-45e4-951b-6b1dfb56c155 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:50.579253 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21b33f5-14b0-45e4-951b-6b1dfb56c155 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:50.580122 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21b33f5-14b0-45e4-951b-6b1dfb56c155 None admin] RBAC: Adding query filter params (interface=public, service_id=c8c19c51a78044e999f0a7f75a8dc715, region_id=RegionOne) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:50.580430 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21b33f5-14b0-45e4-951b-6b1dfb56c155 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:50.581964 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21b33f5-14b0-45e4-951b-6b1dfb56c155 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:50.592539 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 377/757] 10.222.0.36 () {56 vars in 1251 bytes} [Wed Oct 12 00:02:50 2022] GET /identity/v3/endpoints?service_id=c8c19c51a78044e999f0a7f75a8dc715&interface=public®ion_id=RegionOne => generated 194 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:50.593696 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21b33f5-14b0-45e4-951b-6b1dfb56c155 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:50.637976 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:50.651817 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:50.688712 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72636f84-ab99-4757-afec-2264d7919079 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:50.702157 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21b33f5-14b0-45e4-951b-6b1dfb56c155 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:50.729316 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b082f149-4abd-4874-9f3d-3fb14fea4947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:50.730273 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b082f149-4abd-4874-9f3d-3fb14fea4947 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:50.730908 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b082f149-4abd-4874-9f3d-3fb14fea4947 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:50.731615 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b082f149-4abd-4874-9f3d-3fb14fea4947 None admin] PATH_INFO: `/v3/services` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:50.732185 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b082f149-4abd-4874-9f3d-3fb14fea4947 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:50.732492 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b082f149-4abd-4874-9f3d-3fb14fea4947 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:50.733870 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b082f149-4abd-4874-9f3d-3fb14fea4947 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:50.745296 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 381/758] 10.222.0.36 () {56 vars in 1122 bytes} [Wed Oct 12 00:02:50 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:50.746240 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b082f149-4abd-4874-9f3d-3fb14fea4947 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:50.749751 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e26115a-c5e5-41f4-8c24-20dfb946f085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:50.750907 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e26115a-c5e5-41f4-8c24-20dfb946f085 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:50.751184 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e26115a-c5e5-41f4-8c24-20dfb946f085 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:50.751479 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e26115a-c5e5-41f4-8c24-20dfb946f085 None admin] PATH_INFO: `/v3/services` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:50.752735 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e26115a-c5e5-41f4-8c24-20dfb946f085 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:50.753576 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e26115a-c5e5-41f4-8c24-20dfb946f085 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:50.767770 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 378/759] 10.222.0.36 () {56 vars in 1087 bytes} [Wed Oct 12 00:02:50 2022] GET /identity/v3/services => generated 1957 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 00:02:50.861643 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b082f149-4abd-4874-9f3d-3fb14fea4947 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:50.888684 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:50.899432 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8671f863-bbcf-44ee-9a7b-eb3a8ab2bf73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:50.900068 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8671f863-bbcf-44ee-9a7b-eb3a8ab2bf73 None admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:50.900343 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8671f863-bbcf-44ee-9a7b-eb3a8ab2bf73 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:50.900610 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8671f863-bbcf-44ee-9a7b-eb3a8ab2bf73 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:50.901288 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8671f863-bbcf-44ee-9a7b-eb3a8ab2bf73 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:50.906525 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8671f863-bbcf-44ee-9a7b-eb3a8ab2bf73 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:50.983638 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 382/760] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:02:50 2022] POST /identity/v3/endpoints => generated 325 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:02:51.126488 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31443de5-feec-4d6a-8666-e242096aa325 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:51.126844 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31443de5-feec-4d6a-8666-e242096aa325 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:51.127040 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31443de5-feec-4d6a-8666-e242096aa325 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:51.128862 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 379/761] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:02:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:02:51.134947 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7597853a-9532-4f03-8209-3fc730ee2396 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:51.135226 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7597853a-9532-4f03-8209-3fc730ee2396 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:51.135526 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7597853a-9532-4f03-8209-3fc730ee2396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:51.155512 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-7597853a-9532-4f03-8209-3fc730ee2396 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:02:51.158232 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-7597853a-9532-4f03-8209-3fc730ee2396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:02:51.233614 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 383/762] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:02:51 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 00:02:51.503111 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31443de5-feec-4d6a-8666-e242096aa325 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:51.612438 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31443de5-feec-4d6a-8666-e242096aa325 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:51.633325 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:51.653699 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc06374d-6f32-433c-9b7d-7b35fb440793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:51.656908 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc06374d-6f32-433c-9b7d-7b35fb440793 None admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:51.656908 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc06374d-6f32-433c-9b7d-7b35fb440793 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:51.656908 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc06374d-6f32-433c-9b7d-7b35fb440793 None admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:51.656908 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc06374d-6f32-433c-9b7d-7b35fb440793 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:51.657189 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc06374d-6f32-433c-9b7d-7b35fb440793 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:51.710991 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.sql.core [None req-fc06374d-6f32-433c-9b7d-7b35fb440793 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 12 00:02:51.710991 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 12 00:02:51.710991 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [parameters: {'user_id': 'b67c2722a13b4c3abb7c39c9293743f0', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 12 00:02:51.710991 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81746) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 12 00:02:51.718440 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-fc06374d-6f32-433c-9b7d-7b35fb440793 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 12 00:02:51.718440 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:51.718440 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 12 00:02:51.718440 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 12 00:02:51.718440 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 12 00:02:51.718440 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 12 00:02:51.718440 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 12 00:02:51.718440 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application result = self._query(query) Oct 12 00:02:51.718440 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 12 00:02:51.718440 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application conn.query(q) Oct 12 00:02:51.718440 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 12 00:02:51.718440 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 12 00:02:51.718440 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 12 00:02:51.718440 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application result.read() Oct 12 00:02:51.718440 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 12 00:02:51.719348 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 12 00:02:51.719348 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 12 00:02:51.719348 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 12 00:02:51.719348 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 12 00:02:51.719348 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 12 00:02:51.719348 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 12 00:02:51.719348 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 12 00:02:51.719348 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 12 00:02:51.719348 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:51.719348 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 12 00:02:51.719348 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:51.719348 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:51.719348 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 12 00:02:51.719348 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 12 00:02:51.719348 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Oct 12 00:02:51.719348 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Oct 12 00:02:51.719348 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 12 00:02:51.719348 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application next(self.gen) Oct 12 00:02:51.720369 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 12 00:02:51.720369 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application yield resource Oct 12 00:02:51.720369 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 12 00:02:51.720369 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application next(self.gen) Oct 12 00:02:51.720369 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 12 00:02:51.720369 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self.session.rollback() Oct 12 00:02:51.720369 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 12 00:02:51.720369 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self.force_reraise() Oct 12 00:02:51.720369 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 12 00:02:51.720369 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise self.value Oct 12 00:02:51.720369 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 12 00:02:51.720369 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 12 00:02:51.720369 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 12 00:02:51.720369 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application session.commit() Oct 12 00:02:51.720369 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 12 00:02:51.720369 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 12 00:02:51.720369 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 12 00:02:51.721220 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self._prepare_impl() Oct 12 00:02:51.721220 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 12 00:02:51.721220 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self.session.flush() Oct 12 00:02:51.721220 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 12 00:02:51.721220 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self._flush(objects) Oct 12 00:02:51.721220 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 12 00:02:51.721220 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 12 00:02:51.721220 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 12 00:02:51.721220 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application compat.raise_( Oct 12 00:02:51.721220 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 12 00:02:51.721220 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception Oct 12 00:02:51.721220 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 12 00:02:51.721220 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application flush_context.execute() Oct 12 00:02:51.721220 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 12 00:02:51.721220 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rec.execute(self) Oct 12 00:02:51.721220 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 12 00:02:51.721220 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 12 00:02:51.722103 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 12 00:02:51.722103 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 12 00:02:51.722103 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Oct 12 00:02:51.722103 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application result = connection._execute_20( Oct 12 00:02:51.722103 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 12 00:02:51.722103 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 12 00:02:51.722103 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 12 00:02:51.722103 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 12 00:02:51.722103 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 12 00:02:51.722103 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 12 00:02:51.722103 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 12 00:02:51.722103 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 12 00:02:51.722103 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 12 00:02:51.722103 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 12 00:02:51.722103 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 12 00:02:51.722103 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception Oct 12 00:02:51.722954 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 12 00:02:51.722954 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 12 00:02:51.722954 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 12 00:02:51.722954 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 12 00:02:51.722954 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 12 00:02:51.722954 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application result = self._query(query) Oct 12 00:02:51.722954 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 12 00:02:51.722954 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application conn.query(q) Oct 12 00:02:51.722954 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 12 00:02:51.722954 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 12 00:02:51.722954 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 12 00:02:51.722954 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application result.read() Oct 12 00:02:51.722954 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 12 00:02:51.722954 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 12 00:02:51.722954 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 12 00:02:51.722954 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 12 00:02:51.723811 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 12 00:02:51.723811 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 12 00:02:51.723811 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 12 00:02:51.723811 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 12 00:02:51.723811 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 12 00:02:51.723811 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 12 00:02:51.723811 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [parameters: {'user_id': 'b67c2722a13b4c3abb7c39c9293743f0', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 12 00:02:51.723811 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 12 00:02:51.723811 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:51.723811 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 00:02:51.723811 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:51.723811 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:51.723811 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:51.723811 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:51.723811 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:51.724587 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:51.724587 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:51.724587 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:51.724587 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:51.724587 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:51.724587 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:51.724587 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:51.724587 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Oct 12 00:02:51.724587 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Oct 12 00:02:51.724587 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:51.724587 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:51.724587 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 00:02:51.724587 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 00:02:51.724587 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 00:02:51.724587 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 00:02:51.724587 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Oct 12 00:02:51.724587 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Oct 12 00:02:51.725495 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Oct 12 00:02:51.725495 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Oct 12 00:02:51.725495 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 12 00:02:51.725495 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 12 00:02:51.725495 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 12 00:02:51.725495 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:51.725495 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 380/763] 10.222.0.36 () {60 vars in 1131 bytes} [Wed Oct 12 00:02:51 2022] POST /identity/v3/users => generated 160 bytes in 217 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:51.728339 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7597853a-9532-4f03-8209-3fc730ee2396 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:51.752650 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2650ed5-5e5d-41b1-a2b2-c8eab7c29807 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:51.752650 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2650ed5-5e5d-41b1-a2b2-c8eab7c29807 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:51.752650 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2650ed5-5e5d-41b1-a2b2-c8eab7c29807 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:51.759420 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 381/764] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:02:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 00:02:51.760716 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1c8736-09ff-414d-8da9-6fa48a4d3b19 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:51.761136 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1c8736-09ff-414d-8da9-6fa48a4d3b19 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:51.761542 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1c8736-09ff-414d-8da9-6fa48a4d3b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:51.780742 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-5e1c8736-09ff-414d-8da9-6fa48a4d3b19 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:02:51.782529 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-5e1c8736-09ff-414d-8da9-6fa48a4d3b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:02:51.840066 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7597853a-9532-4f03-8209-3fc730ee2396 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:51.849872 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:51.856138 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 382/765] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:02:51 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 00:02:51.877507 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7581ce9-5092-4d9b-bc6f-1fde0383a5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:51.878144 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7581ce9-5092-4d9b-bc6f-1fde0383a5d2 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:51.878419 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7581ce9-5092-4d9b-bc6f-1fde0383a5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:51.878687 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7581ce9-5092-4d9b-bc6f-1fde0383a5d2 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:51.886852 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7581ce9-5092-4d9b-bc6f-1fde0383a5d2 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:51.888905 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7581ce9-5092-4d9b-bc6f-1fde0383a5d2 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:51.900819 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-b7581ce9-5092-4d9b-bc6f-1fde0383a5d2 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 12 00:02:51.900819 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:51.900819 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:51.900819 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:51.900819 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:51.900819 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:51.900819 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:51.900819 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:51.900819 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:51.900819 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:51.900819 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:51.900819 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:51.900819 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 00:02:51.900819 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 00:02:51.900819 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 00:02:51.900819 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 00:02:51.901724 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:51.901724 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:51.901724 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 00:02:51.901724 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 00:02:51.901724 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 00:02:51.901724 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 00:02:51.901724 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:51.901724 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:51.901724 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:51.901724 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:51.901724 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:51.901724 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:51.901724 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:51.901724 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:51.901724 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:51.901724 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:51.901724 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:51.901724 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:51.902649 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:51.902649 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:51.902649 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 00:02:51.902649 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 00:02:51.902649 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 00:02:51.902649 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 00:02:51.902649 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 00:02:51.902649 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 00:02:51.902649 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 12 00:02:51.902649 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:51.902649 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 384/766] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Oct 12 00:02:51 2022] GET /identity/v3/users/cinder => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:51.908181 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1c8736-09ff-414d-8da9-6fa48a4d3b19 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:52.016298 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1c8736-09ff-414d-8da9-6fa48a4d3b19 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:52.025776 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:52.051594 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b3d279-9138-4627-8e51-1872adcd658a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:52.052202 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b3d279-9138-4627-8e51-1872adcd658a None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:52.052477 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b3d279-9138-4627-8e51-1872adcd658a None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:52.052749 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b3d279-9138-4627-8e51-1872adcd658a None admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:52.053443 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b3d279-9138-4627-8e51-1872adcd658a None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:52.053741 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b3d279-9138-4627-8e51-1872adcd658a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:52.055350 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b3d279-9138-4627-8e51-1872adcd658a None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:52.089421 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 383/767] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Oct 12 00:02:51 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:52.130683 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7581ce9-5092-4d9b-bc6f-1fde0383a5d2 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:52.240938 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7581ce9-5092-4d9b-bc6f-1fde0383a5d2 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:52.283885 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41da2c3d-224d-4556-864b-174143c68562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:52.285011 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41da2c3d-224d-4556-864b-174143c68562 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:52.285490 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41da2c3d-224d-4556-864b-174143c68562 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:52.285959 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41da2c3d-224d-4556-864b-174143c68562 None admin] PATH_INFO: `/v3/services/image` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:52.286762 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41da2c3d-224d-4556-864b-174143c68562 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:52.288396 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41da2c3d-224d-4556-864b-174143c68562 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:52.298231 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-41da2c3d-224d-4556-864b-174143c68562 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 12 00:02:52.298231 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:52.298231 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 00:02:52.298231 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 00:02:52.298231 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 00:02:52.298231 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 00:02:52.298231 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 00:02:52.298231 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 00:02:52.298231 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 12 00:02:52.298231 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:52.298231 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 00:02:52.298231 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:52.298231 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:52.298231 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:52.298231 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:52.298231 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:52.298231 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:52.299195 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:52.299195 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:52.299195 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:52.299195 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:52.299195 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:52.299195 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:52.299195 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 00:02:52.299195 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 00:02:52.299195 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 00:02:52.299195 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 00:02:52.299195 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:52.299195 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:52.299195 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:52.299195 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:52.299195 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:52.299195 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:52.300055 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:52.300055 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:52.300055 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:52.300055 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:52.300055 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:52.300055 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:52.300055 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:52.300055 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:52.300055 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:52.300055 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:52.300055 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 00:02:52.300055 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 00:02:52.300055 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 12 00:02:52.300055 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:52.300055 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 385/768] 10.222.0.36 () {56 vars in 1105 bytes} [Wed Oct 12 00:02:52 2022] GET /identity/v3/services/image => generated 86 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:52.307258 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b3d279-9138-4627-8e51-1872adcd658a None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:52.426211 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b3d279-9138-4627-8e51-1872adcd658a None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:52.461579 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fad139-13be-404c-a963-9ee8eb8f8de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:52.462191 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fad139-13be-404c-a963-9ee8eb8f8de0 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:52.462474 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fad139-13be-404c-a963-9ee8eb8f8de0 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:52.462745 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fad139-13be-404c-a963-9ee8eb8f8de0 None admin] PATH_INFO: `/v3/services` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:52.463383 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fad139-13be-404c-a963-9ee8eb8f8de0 None admin] RBAC: Adding query filter params (name=image) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:52.463689 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fad139-13be-404c-a963-9ee8eb8f8de0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:52.465082 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fad139-13be-404c-a963-9ee8eb8f8de0 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:52.470864 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 384/769] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:02:52 2022] GET /identity/v3/services?name=image => generated 122 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:52.476367 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41da2c3d-224d-4556-864b-174143c68562 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:52.508669 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:52.585346 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41da2c3d-224d-4556-864b-174143c68562 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:52.619292 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0082877d-1b39-4685-b2e0-10b38594124f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:52.619924 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0082877d-1b39-4685-b2e0-10b38594124f None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:52.620201 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0082877d-1b39-4685-b2e0-10b38594124f None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:52.620468 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0082877d-1b39-4685-b2e0-10b38594124f None admin] PATH_INFO: `/v3/services` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:52.621091 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0082877d-1b39-4685-b2e0-10b38594124f None admin] RBAC: Adding query filter params (type=image) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:52.621390 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0082877d-1b39-4685-b2e0-10b38594124f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:52.622738 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0082877d-1b39-4685-b2e0-10b38594124f None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:52.627796 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 386/770] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:02:52 2022] GET /identity/v3/services?type=image => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:52.633749 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fad139-13be-404c-a963-9ee8eb8f8de0 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:52.666023 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:52.735475 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fad139-13be-404c-a963-9ee8eb8f8de0 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:52.769744 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeab421-3d36-4faf-94a4-b7d1c8abdb3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:52.770355 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeab421-3d36-4faf-94a4-b7d1c8abdb3d None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:52.770635 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeab421-3d36-4faf-94a4-b7d1c8abdb3d None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:52.770909 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeab421-3d36-4faf-94a4-b7d1c8abdb3d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:52.771633 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeab421-3d36-4faf-94a4-b7d1c8abdb3d None admin] RBAC: Adding query filter params (interface=public, service_id=a52dceca66e2434a9c91068de74319cf, region_id=RegionOne) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:52.771941 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeab421-3d36-4faf-94a4-b7d1c8abdb3d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:52.773309 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeab421-3d36-4faf-94a4-b7d1c8abdb3d None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:52.782961 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 385/771] 10.222.0.36 () {56 vars in 1251 bytes} [Wed Oct 12 00:02:52 2022] GET /identity/v3/endpoints?service_id=a52dceca66e2434a9c91068de74319cf&interface=public®ion_id=RegionOne => generated 194 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 00:02:52.784227 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0082877d-1b39-4685-b2e0-10b38594124f None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:52.881229 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:52.887935 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0082877d-1b39-4685-b2e0-10b38594124f None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:52.922266 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61dc434-3be9-48ae-bbf1-e2752522d2d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:52.923204 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61dc434-3be9-48ae-bbf1-e2752522d2d6 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:52.923532 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61dc434-3be9-48ae-bbf1-e2752522d2d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:52.923801 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61dc434-3be9-48ae-bbf1-e2752522d2d6 None admin] PATH_INFO: `/v3/services` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:52.924401 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61dc434-3be9-48ae-bbf1-e2752522d2d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:52.925765 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61dc434-3be9-48ae-bbf1-e2752522d2d6 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:52.938377 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 387/772] 10.222.0.36 () {56 vars in 1087 bytes} [Wed Oct 12 00:02:52 2022] GET /identity/v3/services => generated 1957 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:02:52.971676 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ba3939-61c6-49b9-a4c9-5675f1aeba19 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:52.971951 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ba3939-61c6-49b9-a4c9-5675f1aeba19 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:52.972225 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ba3939-61c6-49b9-a4c9-5675f1aeba19 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:52.973607 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 386/773] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:02:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:02:52.979426 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f71686-f04b-4fa1-9fb7-eba5e84257be None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:52.979707 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f71686-f04b-4fa1-9fb7-eba5e84257be None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:52.979983 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f71686-f04b-4fa1-9fb7-eba5e84257be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:52.998987 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-91f71686-f04b-4fa1-9fb7-eba5e84257be None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:02:53.000748 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-91f71686-f04b-4fa1-9fb7-eba5e84257be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:02:53.074171 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 388/774] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:02:52 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 00:02:53.301194 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b172b4bd-1665-4b22-bb0a-4c7e2e7c5a40 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:53.301367 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b172b4bd-1665-4b22-bb0a-4c7e2e7c5a40 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:53.301650 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b172b4bd-1665-4b22-bb0a-4c7e2e7c5a40 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:53.303137 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 387/775] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:02:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:02:53.308257 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ffa5e8-0650-4473-989b-f1ea43b35550 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:53.308548 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ffa5e8-0650-4473-989b-f1ea43b35550 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:53.308823 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ffa5e8-0650-4473-989b-f1ea43b35550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:53.327215 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-42ffa5e8-0650-4473-989b-f1ea43b35550 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:02:53.328455 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-42ffa5e8-0650-4473-989b-f1ea43b35550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:02:53.338851 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b172b4bd-1665-4b22-bb0a-4c7e2e7c5a40 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:53.396476 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 389/776] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:02:53 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 00:02:53.437117 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:53.445703 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b172b4bd-1665-4b22-bb0a-4c7e2e7c5a40 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:53.479528 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327bd514-f7fb-41d6-9c63-b7401045cf62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:53.480163 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327bd514-f7fb-41d6-9c63-b7401045cf62 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:53.480446 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327bd514-f7fb-41d6-9c63-b7401045cf62 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:53.480716 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327bd514-f7fb-41d6-9c63-b7401045cf62 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:53.481295 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327bd514-f7fb-41d6-9c63-b7401045cf62 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:53.482658 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327bd514-f7fb-41d6-9c63-b7401045cf62 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:53.492218 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-327bd514-f7fb-41d6-9c63-b7401045cf62 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 12 00:02:53.492218 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:53.492218 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 00:02:53.492218 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 00:02:53.492218 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 00:02:53.492218 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 00:02:53.492218 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 00:02:53.492218 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 00:02:53.492218 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 12 00:02:53.492218 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:53.492218 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 00:02:53.492218 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:53.492218 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:53.492218 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:53.492218 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:53.492218 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:53.492218 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:53.493178 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:53.493178 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:53.493178 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:53.493178 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:53.493178 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:53.493178 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:53.493178 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 00:02:53.493178 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 00:02:53.493178 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 00:02:53.493178 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 00:02:53.493178 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:53.493178 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:53.493178 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:53.493178 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:53.493178 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:53.493178 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:53.493991 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:53.493991 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:53.493991 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:53.493991 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:53.493991 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:53.493991 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:53.493991 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:53.493991 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:53.493991 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:53.493991 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:53.493991 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 00:02:53.493991 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 00:02:53.493991 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 12 00:02:53.493991 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:53.495779 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 388/777] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Oct 12 00:02:53 2022] GET /identity/v3/services/compute => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 00:02:53.499179 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ffa5e8-0650-4473-989b-f1ea43b35550 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:53.520845 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:53.602033 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ffa5e8-0650-4473-989b-f1ea43b35550 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:53.635695 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e086f7-053c-4d52-a8e6-66d186c763a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:53.636302 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e086f7-053c-4d52-a8e6-66d186c763a2 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:53.636576 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e086f7-053c-4d52-a8e6-66d186c763a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:53.636843 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e086f7-053c-4d52-a8e6-66d186c763a2 None admin] PATH_INFO: `/v3/services` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:53.637462 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e086f7-053c-4d52-a8e6-66d186c763a2 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:53.637758 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e086f7-053c-4d52-a8e6-66d186c763a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:53.639115 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e086f7-053c-4d52-a8e6-66d186c763a2 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:53.648598 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 390/778] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Oct 12 00:02:53 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 00:02:53.650120 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327bd514-f7fb-41d6-9c63-b7401045cf62 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:53.663989 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e086f7-053c-4d52-a8e6-66d186c763a2 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:53.753467 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:53.761834 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327bd514-f7fb-41d6-9c63-b7401045cf62 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:53.770119 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e086f7-053c-4d52-a8e6-66d186c763a2 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:53.797660 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d871ac3-6e0f-4f7c-96d8-3c20612b96b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:53.798265 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d871ac3-6e0f-4f7c-96d8-3c20612b96b5 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:53.798545 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d871ac3-6e0f-4f7c-96d8-3c20612b96b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:53.798817 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d871ac3-6e0f-4f7c-96d8-3c20612b96b5 None admin] PATH_INFO: `/v3/services` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:53.799480 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d871ac3-6e0f-4f7c-96d8-3c20612b96b5 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:53.799781 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d871ac3-6e0f-4f7c-96d8-3c20612b96b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:53.801165 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d871ac3-6e0f-4f7c-96d8-3c20612b96b5 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:53.811257 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 389/779] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Oct 12 00:02:53 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:53.813116 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d871ac3-6e0f-4f7c-96d8-3c20612b96b5 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:53.814249 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45224880-2790-44ab-a005-9a13ebd2a6d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:53.814910 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45224880-2790-44ab-a005-9a13ebd2a6d2 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:53.815186 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45224880-2790-44ab-a005-9a13ebd2a6d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:53.815492 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45224880-2790-44ab-a005-9a13ebd2a6d2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:53.816684 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45224880-2790-44ab-a005-9a13ebd2a6d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:53.818145 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45224880-2790-44ab-a005-9a13ebd2a6d2 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:53.820051 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 391/780] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:02:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 00:02:53.825568 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45224880-2790-44ab-a005-9a13ebd2a6d2 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:53.844815 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:53.923872 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d871ac3-6e0f-4f7c-96d8-3c20612b96b5 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:53.932251 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45224880-2790-44ab-a005-9a13ebd2a6d2 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:53.957630 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0cb3f8-bb9e-4736-96b2-bd6e09daf404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:53.958236 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0cb3f8-bb9e-4736-96b2-bd6e09daf404 None admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:53.958516 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0cb3f8-bb9e-4736-96b2-bd6e09daf404 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:53.958785 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0cb3f8-bb9e-4736-96b2-bd6e09daf404 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:53.959473 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0cb3f8-bb9e-4736-96b2-bd6e09daf404 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:53.960848 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0cb3f8-bb9e-4736-96b2-bd6e09daf404 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:53.966585 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eef703c-7a30-4539-903a-d2d3b29b3714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:53.967185 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eef703c-7a30-4539-903a-d2d3b29b3714 None admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:53.967491 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eef703c-7a30-4539-903a-d2d3b29b3714 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:53.967762 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eef703c-7a30-4539-903a-d2d3b29b3714 None admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:53.968398 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eef703c-7a30-4539-903a-d2d3b29b3714 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:53.970438 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eef703c-7a30-4539-903a-d2d3b29b3714 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:54.053975 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 390/781] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:02:53 2022] POST /identity/v3/endpoints => generated 334 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:02:54.067627 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 392/782] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Oct 12 00:02:53 2022] POST /identity/v3/projects => generated 311 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 00:02:55.095396 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b5fc63-d02a-4624-84c4-46d165133d51 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:55.095948 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b5fc63-d02a-4624-84c4-46d165133d51 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:55.095948 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b5fc63-d02a-4624-84c4-46d165133d51 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:55.097294 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 391/783] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:02:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:02:55.102339 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595ca8c1-386f-4aa6-a4fd-63c817e7d7cf None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:55.102737 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595ca8c1-386f-4aa6-a4fd-63c817e7d7cf None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:55.102899 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595ca8c1-386f-4aa6-a4fd-63c817e7d7cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:55.110478 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:55.121055 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-595ca8c1-386f-4aa6-a4fd-63c817e7d7cf None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:02:55.122218 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-595ca8c1-386f-4aa6-a4fd-63c817e7d7cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:02:55.192710 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 393/784] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:02:55 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 00:02:55.459036 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b5fc63-d02a-4624-84c4-46d165133d51 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:55.543194 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:55.566000 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b5fc63-d02a-4624-84c4-46d165133d51 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:55.600678 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34525374-32c0-4680-bbdc-71c541a063d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:55.601977 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34525374-32c0-4680-bbdc-71c541a063d8 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:55.602257 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34525374-32c0-4680-bbdc-71c541a063d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:55.602528 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34525374-32c0-4680-bbdc-71c541a063d8 None admin] PATH_INFO: `/v3/services/image` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:55.603107 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34525374-32c0-4680-bbdc-71c541a063d8 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:55.604550 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34525374-32c0-4680-bbdc-71c541a063d8 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:55.613344 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-34525374-32c0-4680-bbdc-71c541a063d8 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 12 00:02:55.613344 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:55.613344 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 00:02:55.613344 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 00:02:55.613344 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 00:02:55.613344 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 00:02:55.613344 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 00:02:55.613344 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 00:02:55.613344 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 12 00:02:55.613344 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:55.613344 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 00:02:55.613344 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:55.613344 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:55.613344 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:55.613344 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:55.613344 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:55.613344 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:55.614319 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:55.614319 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:55.614319 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:55.614319 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:55.614319 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:55.614319 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:55.614319 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 00:02:55.614319 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 00:02:55.614319 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 00:02:55.614319 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 00:02:55.614319 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:55.614319 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:55.614319 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:55.614319 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:55.614319 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:55.614319 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:55.615241 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:55.615241 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:55.615241 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:55.615241 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:55.615241 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:55.615241 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:55.615241 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:55.615241 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:55.615241 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:55.615241 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:55.615241 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 00:02:55.615241 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 00:02:55.615241 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 12 00:02:55.615241 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:55.615241 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 392/785] 10.222.0.36 () {56 vars in 1105 bytes} [Wed Oct 12 00:02:55 2022] GET /identity/v3/services/image => generated 86 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:55.621397 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595ca8c1-386f-4aa6-a4fd-63c817e7d7cf None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:55.719596 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595ca8c1-386f-4aa6-a4fd-63c817e7d7cf None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:55.747264 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:55.754263 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7625dce9-dc6e-4431-8b6a-a9ed65221da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:55.754876 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7625dce9-dc6e-4431-8b6a-a9ed65221da2 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:55.758937 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7625dce9-dc6e-4431-8b6a-a9ed65221da2 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:55.758937 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7625dce9-dc6e-4431-8b6a-a9ed65221da2 None admin] PATH_INFO: `/v3/services` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:55.758937 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7625dce9-dc6e-4431-8b6a-a9ed65221da2 None admin] RBAC: Adding query filter params (name=image) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:55.758937 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7625dce9-dc6e-4431-8b6a-a9ed65221da2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:55.758937 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7625dce9-dc6e-4431-8b6a-a9ed65221da2 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:55.763436 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 394/786] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:02:55 2022] GET /identity/v3/services?name=image => generated 122 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:55.771098 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34525374-32c0-4680-bbdc-71c541a063d8 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:55.871726 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34525374-32c0-4680-bbdc-71c541a063d8 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:55.906599 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c27235-2a14-48b0-b101-1087d81b4d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:55.907217 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c27235-2a14-48b0-b101-1087d81b4d9b None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:55.907529 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c27235-2a14-48b0-b101-1087d81b4d9b None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:55.907803 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c27235-2a14-48b0-b101-1087d81b4d9b None admin] PATH_INFO: `/v3/services` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:55.908429 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c27235-2a14-48b0-b101-1087d81b4d9b None admin] RBAC: Adding query filter params (type=image) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:55.908731 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c27235-2a14-48b0-b101-1087d81b4d9b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:55.910143 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c27235-2a14-48b0-b101-1087d81b4d9b None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:55.915144 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 393/787] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:02:55 2022] GET /identity/v3/services?type=image => generated 350 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:55.920820 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7625dce9-dc6e-4431-8b6a-a9ed65221da2 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:56.021488 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7625dce9-dc6e-4431-8b6a-a9ed65221da2 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:56.055077 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d50c36-5b5c-4e70-b099-abd7edbd8c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:56.055714 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d50c36-5b5c-4e70-b099-abd7edbd8c62 None admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:56.055992 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d50c36-5b5c-4e70-b099-abd7edbd8c62 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:56.056270 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d50c36-5b5c-4e70-b099-abd7edbd8c62 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:56.056941 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d50c36-5b5c-4e70-b099-abd7edbd8c62 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:56.058304 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d50c36-5b5c-4e70-b099-abd7edbd8c62 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:56.159401 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 395/788] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:02:55 2022] POST /identity/v3/endpoints => generated 327 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:02:56.274419 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6231f800-937f-4705-9fa4-22d3a12cb32f None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:56.274925 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6231f800-937f-4705-9fa4-22d3a12cb32f None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:56.274925 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6231f800-937f-4705-9fa4-22d3a12cb32f None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:56.276478 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 394/789] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:02:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:02:56.282522 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a73849-c481-442f-9420-e06564fc9374 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:56.282806 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a73849-c481-442f-9420-e06564fc9374 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:56.283109 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a73849-c481-442f-9420-e06564fc9374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:56.302116 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-e8a73849-c481-442f-9420-e06564fc9374 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:02:56.303331 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-e8a73849-c481-442f-9420-e06564fc9374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:02:56.315104 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6970127f-c036-4704-9b98-ee9ee607a2b8 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:56.315454 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6970127f-c036-4704-9b98-ee9ee607a2b8 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:56.315684 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6970127f-c036-4704-9b98-ee9ee607a2b8 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:56.319059 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 395/790] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:02:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:02:56.322599 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f104489-251c-40e7-b425-1bd48df94598 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:56.322941 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f104489-251c-40e7-b425-1bd48df94598 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:56.323220 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f104489-251c-40e7-b425-1bd48df94598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:56.336750 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:56.342994 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-7f104489-251c-40e7-b425-1bd48df94598 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:02:56.347520 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-7f104489-251c-40e7-b425-1bd48df94598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:02:56.358223 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:56.390077 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 396/791] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:02:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 00:02:56.420086 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 396/792] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:02:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 12 00:02:56.659518 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a73849-c481-442f-9420-e06564fc9374 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:56.683921 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f104489-251c-40e7-b425-1bd48df94598 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:56.767677 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a73849-c481-442f-9420-e06564fc9374 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:56.791596 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f104489-251c-40e7-b425-1bd48df94598 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:56.804187 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6020359-bb68-4940-8d54-e6dc2bc27792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:56.804750 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6020359-bb68-4940-8d54-e6dc2bc27792 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:56.805020 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6020359-bb68-4940-8d54-e6dc2bc27792 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:56.805283 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6020359-bb68-4940-8d54-e6dc2bc27792 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:56.806508 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6020359-bb68-4940-8d54-e6dc2bc27792 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:56.808037 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6020359-bb68-4940-8d54-e6dc2bc27792 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:56.810110 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 397/793] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:02:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:56.815459 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6020359-bb68-4940-8d54-e6dc2bc27792 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:56.829143 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0823961-cecc-4d9e-b0ce-d405ebcff20b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:56.829803 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0823961-cecc-4d9e-b0ce-d405ebcff20b None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:56.830037 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0823961-cecc-4d9e-b0ce-d405ebcff20b None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:56.830302 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0823961-cecc-4d9e-b0ce-d405ebcff20b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:56.833344 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:56.841075 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0823961-cecc-4d9e-b0ce-d405ebcff20b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 00:02:56.846449 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-d0823961-cecc-4d9e-b0ce-d405ebcff20b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 12 00:02:56.846449 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:56.846449 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:56.846449 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:56.846449 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:56.846449 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:56.846449 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:56.846449 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:56.846449 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:56.846449 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:56.846449 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:56.846449 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:56.846449 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 00:02:56.846449 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 00:02:56.846449 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 00:02:56.846449 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 00:02:56.851474 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 00:02:56.851474 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 00:02:56.851474 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 00:02:56.851474 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 00:02:56.851474 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:56.851474 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:56.851474 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:56.851474 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:56.851474 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:56.851474 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:56.851474 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:56.851474 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:56.851474 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:56.851474 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:56.851474 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:56.851474 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:56.852412 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:56.852412 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:56.852412 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:56.852412 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:56.852412 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 00:02:56.852412 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 00:02:56.852412 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 00:02:56.852412 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 00:02:56.852412 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 00:02:56.852412 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 00:02:56.852412 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 12 00:02:56.852412 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:56.852412 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 397/794] 10.222.0.36 () {56 vars in 1120 bytes} [Wed Oct 12 00:02:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 00:02:56.854011 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0823961-cecc-4d9e-b0ce-d405ebcff20b None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:56.865393 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:56.924035 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6020359-bb68-4940-8d54-e6dc2bc27792 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:56.958798 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506b69e8-a3ea-48ef-bd6c-b0b1e1eaa1c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:56.959031 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0823961-cecc-4d9e-b0ce-d405ebcff20b None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:56.959427 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b69e8-a3ea-48ef-bd6c-b0b1e1eaa1c2 None admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:56.959701 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b69e8-a3ea-48ef-bd6c-b0b1e1eaa1c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:56.959963 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b69e8-a3ea-48ef-bd6c-b0b1e1eaa1c2 None admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:56.960722 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506b69e8-a3ea-48ef-bd6c-b0b1e1eaa1c2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:56.962271 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506b69e8-a3ea-48ef-bd6c-b0b1e1eaa1c2 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:56.993155 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93580acf-9587-4645-a36b-88f253830532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:56.993762 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93580acf-9587-4645-a36b-88f253830532 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:56.994041 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93580acf-9587-4645-a36b-88f253830532 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:56.994313 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93580acf-9587-4645-a36b-88f253830532 None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:56.994974 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93580acf-9587-4645-a36b-88f253830532 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:56.995277 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93580acf-9587-4645-a36b-88f253830532 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:56.996802 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93580acf-9587-4645-a36b-88f253830532 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:57.004689 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 398/795] 10.222.0.36 () {56 vars in 1115 bytes} [Wed Oct 12 00:02:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:57.010487 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93580acf-9587-4645-a36b-88f253830532 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:57.069226 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:57.072043 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 398/796] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Oct 12 00:02:56 2022] POST /identity/v3/users => generated 285 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 00:02:57.122261 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93580acf-9587-4645-a36b-88f253830532 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:57.160678 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa8e404-8683-4c78-907b-78deae249fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:57.161331 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa8e404-8683-4c78-907b-78deae249fab None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:57.161613 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa8e404-8683-4c78-907b-78deae249fab None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:57.161886 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa8e404-8683-4c78-907b-78deae249fab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:57.173704 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa8e404-8683-4c78-907b-78deae249fab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:57.175186 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa8e404-8683-4c78-907b-78deae249fab None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:57.190467 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-3fa8e404-8683-4c78-907b-78deae249fab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:57.190467 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:57.190467 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:57.190467 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:57.190467 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:57.190467 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:57.190467 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:57.190467 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:57.190467 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:57.190467 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:57.190467 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:57.190467 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:57.190467 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 00:02:57.190467 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 00:02:57.190467 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 00:02:57.190467 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 00:02:57.191571 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:57.191571 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:57.191571 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:57.191571 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:57.191571 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:57.191571 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:57.191571 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:57.191571 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:57.191571 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:57.191571 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:57.191571 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:57.191571 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:57.191571 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:57.191571 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:57.191571 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:57.191571 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:57.191571 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 00:02:57.192486 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 00:02:57.192486 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:57.192486 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:57.192486 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 399/797] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:02:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 00:02:57.197683 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506b69e8-a3ea-48ef-bd6c-b0b1e1eaa1c2 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:57.300492 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:57.303774 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506b69e8-a3ea-48ef-bd6c-b0b1e1eaa1c2 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:57.338900 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e25431-2efe-456c-b348-7b1fd9d65a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:57.339544 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e25431-2efe-456c-b348-7b1fd9d65a1c None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:57.339823 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e25431-2efe-456c-b348-7b1fd9d65a1c None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:57.340091 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e25431-2efe-456c-b348-7b1fd9d65a1c None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:57.340731 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e25431-2efe-456c-b348-7b1fd9d65a1c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:57.341031 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e25431-2efe-456c-b348-7b1fd9d65a1c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:57.342390 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e25431-2efe-456c-b348-7b1fd9d65a1c None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:57.358153 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 399/798] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Oct 12 00:02:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:57.359163 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa8e404-8683-4c78-907b-78deae249fab None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:57.461693 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa8e404-8683-4c78-907b-78deae249fab None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:57.472263 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:57.495973 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5427f6fa-440e-4b28-a659-e3a4397c9ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:57.496616 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5427f6fa-440e-4b28-a659-e3a4397c9ada None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:57.496893 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5427f6fa-440e-4b28-a659-e3a4397c9ada None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:57.497210 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5427f6fa-440e-4b28-a659-e3a4397c9ada None admin] PATH_INFO: `/v3/users/nova` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:57.505835 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5427f6fa-440e-4b28-a659-e3a4397c9ada None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:57.507268 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5427f6fa-440e-4b28-a659-e3a4397c9ada None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:57.519164 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-5427f6fa-440e-4b28-a659-e3a4397c9ada None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 12 00:02:57.519164 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:57.519164 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:57.519164 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:57.519164 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:57.519164 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:57.519164 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:57.519164 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:57.519164 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:57.519164 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:57.519164 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:57.519164 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:57.519164 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 00:02:57.519164 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 00:02:57.519164 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 00:02:57.519164 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 00:02:57.520234 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:57.520234 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:57.520234 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 00:02:57.520234 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 00:02:57.520234 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 00:02:57.520234 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 00:02:57.520234 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:57.520234 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:57.520234 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:57.520234 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:57.520234 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:57.520234 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:57.520234 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:57.520234 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:57.520234 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:57.520234 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:57.520234 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:57.520234 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:57.521145 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:57.521145 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:57.521145 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 00:02:57.521145 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 00:02:57.521145 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 00:02:57.521145 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 00:02:57.521145 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 00:02:57.521145 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 00:02:57.521145 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 12 00:02:57.521145 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:57.523452 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 400/799] 10.222.0.36 () {56 vars in 1093 bytes} [Wed Oct 12 00:02:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:57.526145 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e25431-2efe-456c-b348-7b1fd9d65a1c None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:57.632572 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e25431-2efe-456c-b348-7b1fd9d65a1c None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:57.665107 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f0a8bd-5e19-46f7-80bf-8ca6d6d684aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:57.665711 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f0a8bd-5e19-46f7-80bf-8ca6d6d684aa None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:57.665988 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f0a8bd-5e19-46f7-80bf-8ca6d6d684aa None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:57.666255 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f0a8bd-5e19-46f7-80bf-8ca6d6d684aa None admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:57.666975 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f0a8bd-5e19-46f7-80bf-8ca6d6d684aa None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:57.667273 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f0a8bd-5e19-46f7-80bf-8ca6d6d684aa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:57.668734 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f0a8bd-5e19-46f7-80bf-8ca6d6d684aa None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:57.686080 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 400/800] 10.222.0.36 () {56 vars in 1133 bytes} [Wed Oct 12 00:02:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:57.692245 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5427f6fa-440e-4b28-a659-e3a4397c9ada None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:57.797488 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5427f6fa-440e-4b28-a659-e3a4397c9ada None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:57.831537 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bab234-bd56-4844-9dba-6f7b4866cca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:57.832173 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bab234-bd56-4844-9dba-6f7b4866cca3 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:57.832449 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bab234-bd56-4844-9dba-6f7b4866cca3 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:57.832717 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bab234-bd56-4844-9dba-6f7b4866cca3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:57.843310 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bab234-bd56-4844-9dba-6f7b4866cca3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:57.844775 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bab234-bd56-4844-9dba-6f7b4866cca3 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:57.859348 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-79bab234-bd56-4844-9dba-6f7b4866cca3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:57.859348 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:57.859348 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:57.859348 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:57.859348 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:57.859348 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:57.859348 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:57.859348 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:57.859348 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:57.859348 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:57.859348 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:57.859348 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:57.859348 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 00:02:57.859348 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 00:02:57.859348 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 00:02:57.859348 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 00:02:57.860312 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:57.860312 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:57.860312 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:57.860312 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:57.860312 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:57.860312 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:57.860312 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:57.860312 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:57.860312 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:57.860312 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:57.860312 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:57.860312 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:57.860312 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:57.860312 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:57.860312 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:57.860312 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:57.860312 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 00:02:57.865870 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 00:02:57.865870 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:57.865870 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:57.865870 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 401/801] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:02:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 00:02:57.867642 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f0a8bd-5e19-46f7-80bf-8ca6d6d684aa None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:57.888361 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:57.967842 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f0a8bd-5e19-46f7-80bf-8ca6d6d684aa None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:58.004832 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9f525f-fef6-439a-a1d7-98ec7e6801e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:58.005453 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9f525f-fef6-439a-a1d7-98ec7e6801e2 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:58.005730 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9f525f-fef6-439a-a1d7-98ec7e6801e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:58.006000 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9f525f-fef6-439a-a1d7-98ec7e6801e2 None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:58.006638 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9f525f-fef6-439a-a1d7-98ec7e6801e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:58.006935 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9f525f-fef6-439a-a1d7-98ec7e6801e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:58.008584 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9f525f-fef6-439a-a1d7-98ec7e6801e2 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:58.020056 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 401/802] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Oct 12 00:02:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:58.025465 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bab234-bd56-4844-9dba-6f7b4866cca3 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:58.051537 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:58.127390 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bab234-bd56-4844-9dba-6f7b4866cca3 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:58.160977 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30979706-306f-4fe0-82c5-ff88be3884f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:58.161543 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30979706-306f-4fe0-82c5-ff88be3884f0 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:58.161824 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30979706-306f-4fe0-82c5-ff88be3884f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:58.162095 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30979706-306f-4fe0-82c5-ff88be3884f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:58.168697 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30979706-306f-4fe0-82c5-ff88be3884f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:58.170142 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30979706-306f-4fe0-82c5-ff88be3884f0 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:58.181762 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-30979706-306f-4fe0-82c5-ff88be3884f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 00:02:58.181762 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:58.181762 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:58.181762 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:58.181762 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:58.181762 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:58.181762 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:58.181762 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:58.181762 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:58.181762 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:58.181762 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:58.181762 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:58.181762 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 00:02:58.181762 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 00:02:58.181762 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 00:02:58.182648 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 00:02:58.182648 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:58.182648 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:58.182648 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:58.182648 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:58.182648 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:58.182648 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:58.182648 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:58.182648 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:58.182648 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:58.182648 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:58.182648 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:58.182648 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:58.182648 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:58.182648 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:58.182648 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:58.182648 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:58.183559 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 00:02:58.183559 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 00:02:58.183559 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 00:02:58.183559 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 00:02:58.183559 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 00:02:58.183559 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 00:02:58.183559 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 00:02:58.183559 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:58.187683 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 402/803] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Oct 12 00:02:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:58.188732 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9f525f-fef6-439a-a1d7-98ec7e6801e2 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:58.273440 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:58.296043 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9f525f-fef6-439a-a1d7-98ec7e6801e2 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:58.331686 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56ca262-3f0c-45b9-96db-a37429f04db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:58.332537 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ca262-3f0c-45b9-96db-a37429f04db9 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:58.332955 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ca262-3f0c-45b9-96db-a37429f04db9 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:58.333365 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ca262-3f0c-45b9-96db-a37429f04db9 None admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:58.334208 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56ca262-3f0c-45b9-96db-a37429f04db9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:58.334650 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56ca262-3f0c-45b9-96db-a37429f04db9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:58.336245 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56ca262-3f0c-45b9-96db-a37429f04db9 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:58.338161 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf2cdd-1dac-40ec-bc04-ce064550cad9 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:58.338448 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf2cdd-1dac-40ec-bc04-ce064550cad9 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:58.338721 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf2cdd-1dac-40ec-bc04-ce064550cad9 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:58.340086 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 403/804] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:02:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:02:58.345479 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b661628e-6dcb-4f38-a47a-d4664bcfa4fa None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:58.345761 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b661628e-6dcb-4f38-a47a-d4664bcfa4fa None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:58.346037 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b661628e-6dcb-4f38-a47a-d4664bcfa4fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:58.350858 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 402/805] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Oct 12 00:02:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 00:02:58.356354 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56ca262-3f0c-45b9-96db-a37429f04db9 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:58.364875 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-b661628e-6dcb-4f38-a47a-d4664bcfa4fa None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:02:58.366580 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-b661628e-6dcb-4f38-a47a-d4664bcfa4fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:02:58.436014 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 404/806] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:02:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 12 00:02:58.461871 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56ca262-3f0c-45b9-96db-a37429f04db9 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:58.495703 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d210696-adb8-413b-b061-477a745f3aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:58.496348 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d210696-adb8-413b-b061-477a745f3aa2 None admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:58.496617 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d210696-adb8-413b-b061-477a745f3aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:58.496879 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d210696-adb8-413b-b061-477a745f3aa2 None admin] PATH_INFO: `/v3/projects/0697927c6aac43d4b14a750fbbb1145e/users/a515b75aa3944912ad2364ffd7924f40/roles/59b6984c4af34bffa59fdac6d215a6a3` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:58.498991 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d210696-adb8-413b-b061-477a745f3aa2 None admin] RBAC: Authorizing `identity:create_grant(project_id=0697927c6aac43d4b14a750fbbb1145e, user_id=a515b75aa3944912ad2364ffd7924f40, role_id=59b6984c4af34bffa59fdac6d215a6a3)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:58.500983 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d210696-adb8-413b-b061-477a745f3aa2 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:58.588799 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:58.594703 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 403/807] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Oct 12 00:02:58 2022] PUT /identity/v3/projects/0697927c6aac43d4b14a750fbbb1145e/users/a515b75aa3944912ad2364ffd7924f40/roles/59b6984c4af34bffa59fdac6d215a6a3 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 00:02:58.703211 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b661628e-6dcb-4f38-a47a-d4664bcfa4fa None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:58.719355 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:58.811353 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b661628e-6dcb-4f38-a47a-d4664bcfa4fa None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:58.845573 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205c9450-a8c3-4b7a-87eb-aeff45246140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:58.846182 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205c9450-a8c3-4b7a-87eb-aeff45246140 None admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:58.846458 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205c9450-a8c3-4b7a-87eb-aeff45246140 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:58.846729 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205c9450-a8c3-4b7a-87eb-aeff45246140 None admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:58.847416 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205c9450-a8c3-4b7a-87eb-aeff45246140 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:58.848785 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205c9450-a8c3-4b7a-87eb-aeff45246140 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:58.899107 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.sql.core [None req-205c9450-a8c3-4b7a-87eb-aeff45246140 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 12 00:02:58.899107 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 12 00:02:58.899107 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [parameters: {'id': '31bdb2a89369411398a93bc17feef4fe', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 12 00:02:58.899107 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81746) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 12 00:02:58.907219 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-205c9450-a8c3-4b7a-87eb-aeff45246140 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 12 00:02:58.907219 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:58.907219 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 12 00:02:58.907219 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 12 00:02:58.907219 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 12 00:02:58.907219 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 12 00:02:58.907219 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 12 00:02:58.907219 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application result = self._query(query) Oct 12 00:02:58.907219 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 12 00:02:58.907219 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application conn.query(q) Oct 12 00:02:58.907219 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 12 00:02:58.907219 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 12 00:02:58.907219 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 12 00:02:58.907219 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application result.read() Oct 12 00:02:58.908117 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 12 00:02:58.908117 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 12 00:02:58.908117 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 12 00:02:58.908117 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 12 00:02:58.908117 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 12 00:02:58.908117 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 12 00:02:58.908117 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 12 00:02:58.908117 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 12 00:02:58.908117 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 12 00:02:58.908117 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:58.908117 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 12 00:02:58.908117 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:58.908117 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:58.908117 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 12 00:02:58.908117 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 12 00:02:58.908117 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 12 00:02:58.908117 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 12 00:02:58.908117 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 12 00:02:58.909081 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application next(self.gen) Oct 12 00:02:58.909081 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 12 00:02:58.909081 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application yield resource Oct 12 00:02:58.909081 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 12 00:02:58.909081 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application next(self.gen) Oct 12 00:02:58.909081 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 12 00:02:58.909081 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self.session.rollback() Oct 12 00:02:58.909081 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 12 00:02:58.909081 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self.force_reraise() Oct 12 00:02:58.909081 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 12 00:02:58.909081 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise self.value Oct 12 00:02:58.909081 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 12 00:02:58.909081 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 12 00:02:58.909081 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 12 00:02:58.909081 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application session.commit() Oct 12 00:02:58.909081 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 12 00:02:58.909081 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 12 00:02:58.909936 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 12 00:02:58.909936 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self._prepare_impl() Oct 12 00:02:58.909936 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 12 00:02:58.909936 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self.session.flush() Oct 12 00:02:58.909936 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 12 00:02:58.909936 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self._flush(objects) Oct 12 00:02:58.909936 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 12 00:02:58.909936 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 12 00:02:58.909936 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 12 00:02:58.909936 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application compat.raise_( Oct 12 00:02:58.909936 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 12 00:02:58.909936 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception Oct 12 00:02:58.909936 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 12 00:02:58.909936 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application flush_context.execute() Oct 12 00:02:58.909936 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 12 00:02:58.909936 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rec.execute(self) Oct 12 00:02:58.909936 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 12 00:02:58.910845 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 12 00:02:58.910845 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 12 00:02:58.910845 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 12 00:02:58.910845 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 12 00:02:58.910845 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 12 00:02:58.910845 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 12 00:02:58.910845 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 12 00:02:58.910845 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 12 00:02:58.910845 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 12 00:02:58.910845 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 12 00:02:58.910845 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 12 00:02:58.910845 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 12 00:02:58.910845 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 12 00:02:58.910845 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 12 00:02:58.910845 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 12 00:02:58.912825 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 12 00:02:58.912825 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception Oct 12 00:02:58.912825 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 12 00:02:58.912825 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 12 00:02:58.912825 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 12 00:02:58.912825 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 12 00:02:58.912825 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 12 00:02:58.912825 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application result = self._query(query) Oct 12 00:02:58.912825 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 12 00:02:58.912825 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application conn.query(q) Oct 12 00:02:58.912825 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 12 00:02:58.912825 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 12 00:02:58.912825 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 12 00:02:58.912825 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application result.read() Oct 12 00:02:58.912825 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 12 00:02:58.912825 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 12 00:02:58.912825 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 12 00:02:58.915026 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 12 00:02:58.915026 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 12 00:02:58.915026 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 12 00:02:58.915026 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 12 00:02:58.915026 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 12 00:02:58.915026 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 12 00:02:58.915026 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 12 00:02:58.915026 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [parameters: {'id': '31bdb2a89369411398a93bc17feef4fe', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 12 00:02:58.915026 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 12 00:02:58.915026 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:58.915026 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 00:02:58.915026 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:58.915026 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:58.915026 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 12 00:02:58.915026 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 12 00:02:58.916921 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 12 00:02:58.916921 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 12 00:02:58.916921 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 12 00:02:58.916921 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:58.916921 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 00:02:58.916921 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:58.916921 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:58.916921 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:58.916921 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:58.916921 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:58.916921 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:58.916921 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:58.916921 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:58.916921 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:58.916921 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:58.916921 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:58.916921 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:58.918151 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 12 00:02:58.918151 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 12 00:02:58.918151 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:58.918151 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:58.918151 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 12 00:02:58.918151 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 12 00:02:58.918151 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:58.918151 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:58.918151 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 12 00:02:58.918151 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 12 00:02:58.918151 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 12 00:02:58.918151 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:02:58.918151 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 405/808] 10.222.0.36 () {60 vars in 1137 bytes} [Wed Oct 12 00:02:58 2022] POST /identity/v3/domains => generated 190 bytes in 206 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:58.918988 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d210696-adb8-413b-b061-477a745f3aa2 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:59.016925 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d210696-adb8-413b-b061-477a745f3aa2 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:59.050530 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c371c4-2f92-4061-aca9-55aae29f7d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:59.051164 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c371c4-2f92-4061-aca9-55aae29f7d13 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:59.051468 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c371c4-2f92-4061-aca9-55aae29f7d13 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:59.051738 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c371c4-2f92-4061-aca9-55aae29f7d13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:59.063351 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c371c4-2f92-4061-aca9-55aae29f7d13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:59.064820 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c371c4-2f92-4061-aca9-55aae29f7d13 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:59.079734 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-45c371c4-2f92-4061-aca9-55aae29f7d13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:59.079734 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:59.079734 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:59.079734 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:59.079734 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:59.079734 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:59.079734 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:59.079734 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:59.079734 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:59.079734 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:59.079734 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:59.079734 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:59.079734 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 00:02:59.079734 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 00:02:59.079734 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 00:02:59.079734 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 00:02:59.080724 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:59.080724 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:59.080724 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:59.080724 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:59.080724 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:59.080724 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:59.080724 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:59.080724 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:59.080724 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:59.080724 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:59.080724 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:59.080724 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:59.080724 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:59.080724 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:59.080724 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:59.080724 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:59.080724 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 00:02:59.081606 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 00:02:59.081606 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:02:59.081606 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:59.083714 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 404/809] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:02:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:59.086751 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205c9450-a8c3-4b7a-87eb-aeff45246140 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:59.170842 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:59.193614 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205c9450-a8c3-4b7a-87eb-aeff45246140 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:59.227938 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54895c3b-105d-4357-868e-552550e31fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:59.228548 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54895c3b-105d-4357-868e-552550e31fac None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:59.228819 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54895c3b-105d-4357-868e-552550e31fac None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:59.229084 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54895c3b-105d-4357-868e-552550e31fac None admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:59.229729 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54895c3b-105d-4357-868e-552550e31fac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:59.230025 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54895c3b-105d-4357-868e-552550e31fac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:59.231402 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54895c3b-105d-4357-868e-552550e31fac None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:59.247405 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 406/810] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Oct 12 00:02:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:59.259045 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd191024-66ff-4e51-8e52-82fe279bde8a None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:59.259283 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd191024-66ff-4e51-8e52-82fe279bde8a None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:59.259611 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd191024-66ff-4e51-8e52-82fe279bde8a None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:59.267443 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 405/811] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:02:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:02:59.267574 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49bf28f-52a6-4731-a262-7faff355a28f None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:59.267653 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49bf28f-52a6-4731-a262-7faff355a28f None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:59.267994 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49bf28f-52a6-4731-a262-7faff355a28f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:59.287988 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-c49bf28f-52a6-4731-a262-7faff355a28f None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:02:59.289199 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-c49bf28f-52a6-4731-a262-7faff355a28f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:02:59.361378 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 407/812] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:02:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 00:02:59.629996 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd191024-66ff-4e51-8e52-82fe279bde8a None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:59.680319 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:59.730755 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd191024-66ff-4e51-8e52-82fe279bde8a None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:59.763585 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d988ca7d-6f6e-4a39-a37c-a1b965ccf42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:59.764205 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d988ca7d-6f6e-4a39-a37c-a1b965ccf42b None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:59.764482 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d988ca7d-6f6e-4a39-a37c-a1b965ccf42b None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:59.764749 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d988ca7d-6f6e-4a39-a37c-a1b965ccf42b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:59.775068 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d988ca7d-6f6e-4a39-a37c-a1b965ccf42b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 00:02:59.779865 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-d988ca7d-6f6e-4a39-a37c-a1b965ccf42b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 00:02:59.779865 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:02:59.779865 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:02:59.779865 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:02:59.779865 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:02:59.779865 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:02:59.779865 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:02:59.779865 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:02:59.779865 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:02:59.779865 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:02:59.779865 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:02:59.779865 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:02:59.779865 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 00:02:59.779865 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 00:02:59.779865 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 00:02:59.779865 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 00:02:59.783819 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 00:02:59.783819 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 00:02:59.783819 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 00:02:59.783819 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 00:02:59.783819 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:02:59.783819 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:02:59.783819 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:02:59.783819 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:02:59.783819 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:02:59.783819 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:02:59.783819 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:02:59.783819 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:02:59.783819 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:02:59.783819 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:02:59.783819 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:02:59.783819 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:02:59.784861 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:02:59.784861 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:02:59.784861 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:02:59.784861 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:02:59.784861 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 00:02:59.784861 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 00:02:59.784861 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 00:02:59.784861 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 00:02:59.784861 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 00:02:59.784861 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 00:02:59.784861 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 00:02:59.784861 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:02:59.784861 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 406/813] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Oct 12 00:02:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:02:59.787024 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49bf28f-52a6-4731-a262-7faff355a28f None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:02:59.851273 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:02:59.886556 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49bf28f-52a6-4731-a262-7faff355a28f None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:02:59.918476 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941260ef-1d29-44ba-aa0b-920f5cce8803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:02:59.919083 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941260ef-1d29-44ba-aa0b-920f5cce8803 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:02:59.919359 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941260ef-1d29-44ba-aa0b-920f5cce8803 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:02:59.919655 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941260ef-1d29-44ba-aa0b-920f5cce8803 None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:02:59.920307 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941260ef-1d29-44ba-aa0b-920f5cce8803 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:02:59.920605 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941260ef-1d29-44ba-aa0b-920f5cce8803 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:02:59.922458 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941260ef-1d29-44ba-aa0b-920f5cce8803 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:02:59.930652 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 408/814] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Oct 12 00:02:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:02:59.936407 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d988ca7d-6f6e-4a39-a37c-a1b965ccf42b None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:03:00.032038 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d988ca7d-6f6e-4a39-a37c-a1b965ccf42b None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:03:00.042214 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:03:00.065261 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4915b6b-0468-4e64-819a-f82b4603e0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:03:00.065880 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4915b6b-0468-4e64-819a-f82b4603e0ab None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:00.066156 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4915b6b-0468-4e64-819a-f82b4603e0ab None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:00.066425 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4915b6b-0468-4e64-819a-f82b4603e0ab None admin] PATH_INFO: `/v3/users/010943d8107746cea1ced6743918667a` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:00.079111 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4915b6b-0468-4e64-819a-f82b4603e0ab None admin] RBAC: Authorizing `identity:get_user(user_id=010943d8107746cea1ced6743918667a)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:03:00.080578 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4915b6b-0468-4e64-819a-f82b4603e0ab None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:03:00.082162 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 407/815] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Oct 12 00:02:59 2022] GET /identity/v3/users/010943d8107746cea1ced6743918667a => generated 285 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 00:03:00.087844 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941260ef-1d29-44ba-aa0b-920f5cce8803 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:03:00.186792 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941260ef-1d29-44ba-aa0b-920f5cce8803 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:03:00.212990 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:03:00.221591 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07b07b2-7a55-4cf6-8168-796425a7efb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:03:00.222251 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b07b2-7a55-4cf6-8168-796425a7efb6 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:00.222530 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b07b2-7a55-4cf6-8168-796425a7efb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:00.222800 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b07b2-7a55-4cf6-8168-796425a7efb6 None admin] PATH_INFO: `/v3/projects/8aea0d97fb5a489a9e29c789b91976f1` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:00.223951 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07b07b2-7a55-4cf6-8168-796425a7efb6 None admin] RBAC: Authorizing `identity:get_project(project_id=8aea0d97fb5a489a9e29c789b91976f1)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:03:00.225376 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07b07b2-7a55-4cf6-8168-796425a7efb6 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:03:00.227016 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 409/816] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Oct 12 00:03:00 2022] GET /identity/v3/projects/8aea0d97fb5a489a9e29c789b91976f1 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:03:00.232867 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4915b6b-0468-4e64-819a-f82b4603e0ab None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:03:00.330799 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4915b6b-0468-4e64-819a-f82b4603e0ab None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:03:00.364092 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4013259b-05ed-4049-b80a-1964463af23f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:03:00.364812 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4013259b-05ed-4049-b80a-1964463af23f None admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:00.365090 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4013259b-05ed-4049-b80a-1964463af23f None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:00.365359 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4013259b-05ed-4049-b80a-1964463af23f None admin] PATH_INFO: `/v3/projects/8aea0d97fb5a489a9e29c789b91976f1/users/010943d8107746cea1ced6743918667a/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:00.367241 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4013259b-05ed-4049-b80a-1964463af23f None admin] RBAC: Authorizing `identity:create_grant(project_id=8aea0d97fb5a489a9e29c789b91976f1, user_id=010943d8107746cea1ced6743918667a, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:03:00.368701 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4013259b-05ed-4049-b80a-1964463af23f None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:03:00.446790 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 408/817] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Oct 12 00:03:00 2022] PUT /identity/v3/projects/8aea0d97fb5a489a9e29c789b91976f1/users/010943d8107746cea1ced6743918667a/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 00:03:00.738938 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae6215c-6dda-4248-bc5a-8085be7b6114 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:00.739122 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae6215c-6dda-4248-bc5a-8085be7b6114 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:00.739440 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae6215c-6dda-4248-bc5a-8085be7b6114 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:00.740834 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 410/818] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:03:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:03:00.746195 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0315a71d-cdf3-424d-be3d-0a4e36ef1407 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:00.746666 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0315a71d-cdf3-424d-be3d-0a4e36ef1407 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:00.746870 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0315a71d-cdf3-424d-be3d-0a4e36ef1407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:00.765416 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-0315a71d-cdf3-424d-be3d-0a4e36ef1407 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:03:00.766595 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-0315a71d-cdf3-424d-be3d-0a4e36ef1407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:03:00.789061 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:03:00.831849 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 409/819] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:03:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 00:03:01.101583 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae6215c-6dda-4248-bc5a-8085be7b6114 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:03:01.199970 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae6215c-6dda-4248-bc5a-8085be7b6114 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:03:01.232874 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7b770b-f6c9-4bf1-a241-b4680abafad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:03:01.233499 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7b770b-f6c9-4bf1-a241-b4680abafad5 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:01.234327 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7b770b-f6c9-4bf1-a241-b4680abafad5 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:01.234596 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7b770b-f6c9-4bf1-a241-b4680abafad5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:01.244723 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7b770b-f6c9-4bf1-a241-b4680abafad5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 00:03:01.249507 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-9f7b770b-f6c9-4bf1-a241-b4680abafad5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 12 00:03:01.249507 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:03:01.249507 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:03:01.249507 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:03:01.249507 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:03:01.249507 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:03:01.249507 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:03:01.249507 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:03:01.249507 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:03:01.249507 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:03:01.249507 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:03:01.249507 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:03:01.249507 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 00:03:01.249507 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 00:03:01.249507 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 00:03:01.249507 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 00:03:01.251166 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 00:03:01.251166 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 00:03:01.251166 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 00:03:01.251166 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 00:03:01.251166 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:03:01.251166 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:03:01.251166 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:03:01.251166 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:03:01.251166 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:03:01.251166 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:03:01.251166 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:03:01.251166 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:03:01.251166 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:03:01.251166 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:03:01.251166 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:03:01.251166 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:03:01.252144 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:03:01.252144 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:03:01.252144 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:03:01.252144 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:03:01.252144 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 00:03:01.252144 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 00:03:01.252144 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 00:03:01.252144 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 00:03:01.252144 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 00:03:01.252144 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 00:03:01.252144 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 12 00:03:01.252144 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:03:01.252144 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 411/820] 10.222.0.36 () {56 vars in 1120 bytes} [Wed Oct 12 00:03:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:03:01.256562 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0315a71d-cdf3-424d-be3d-0a4e36ef1407 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:03:01.355028 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0315a71d-cdf3-424d-be3d-0a4e36ef1407 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:03:01.387003 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b07972-9000-4df5-a93a-6e6cdbd8a7d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:03:01.387631 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b07972-9000-4df5-a93a-6e6cdbd8a7d6 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:01.387907 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b07972-9000-4df5-a93a-6e6cdbd8a7d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:01.388173 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b07972-9000-4df5-a93a-6e6cdbd8a7d6 None admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:01.388823 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b07972-9000-4df5-a93a-6e6cdbd8a7d6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:03:01.389121 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b07972-9000-4df5-a93a-6e6cdbd8a7d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:03:01.390995 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b07972-9000-4df5-a93a-6e6cdbd8a7d6 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:03:01.398589 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 410/821] 10.222.0.36 () {56 vars in 1115 bytes} [Wed Oct 12 00:03:01 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:03:01.404521 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7b770b-f6c9-4bf1-a241-b4680abafad5 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:03:01.460732 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:03:01.502682 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7b770b-f6c9-4bf1-a241-b4680abafad5 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:03:01.538614 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1e4a26-68e2-456c-b072-7c6431943415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:03:01.538614 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e4a26-68e2-456c-b072-7c6431943415 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:01.538614 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e4a26-68e2-456c-b072-7c6431943415 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:01.538614 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e4a26-68e2-456c-b072-7c6431943415 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:01.550793 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1e4a26-68e2-456c-b072-7c6431943415 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:03:01.552276 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1e4a26-68e2-456c-b072-7c6431943415 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:03:01.566506 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-ca1e4a26-68e2-456c-b072-7c6431943415 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:03:01.566506 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:03:01.566506 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:03:01.566506 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:03:01.566506 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:03:01.566506 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:03:01.566506 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:03:01.566506 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:03:01.566506 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:03:01.566506 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:03:01.566506 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:03:01.566506 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:03:01.566506 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 00:03:01.566506 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 00:03:01.566506 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 00:03:01.566506 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 00:03:01.572351 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:03:01.572351 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:03:01.572351 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:03:01.572351 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:03:01.572351 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:03:01.572351 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:03:01.572351 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:03:01.572351 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:03:01.572351 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:03:01.572351 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:03:01.572351 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:03:01.572351 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:03:01.572351 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:03:01.572351 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:03:01.572351 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:03:01.572351 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:03:01.572351 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 00:03:01.573536 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 00:03:01.573536 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:03:01.573536 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:03:01.573536 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 412/822] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:03:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:03:01.573831 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b07972-9000-4df5-a93a-6e6cdbd8a7d6 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:03:01.596210 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:03:01.672402 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b07972-9000-4df5-a93a-6e6cdbd8a7d6 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:03:01.705206 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c366d28e-56dd-460a-b762-2b3410d5e468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:03:01.705805 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c366d28e-56dd-460a-b762-2b3410d5e468 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:01.706075 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c366d28e-56dd-460a-b762-2b3410d5e468 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:01.706339 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c366d28e-56dd-460a-b762-2b3410d5e468 None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:01.706964 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c366d28e-56dd-460a-b762-2b3410d5e468 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:03:01.707257 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c366d28e-56dd-460a-b762-2b3410d5e468 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:03:01.708629 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c366d28e-56dd-460a-b762-2b3410d5e468 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:03:01.720250 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 411/823] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Oct 12 00:03:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:03:01.725603 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1e4a26-68e2-456c-b072-7c6431943415 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:03:01.821183 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1e4a26-68e2-456c-b072-7c6431943415 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:03:01.830091 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:03:01.853355 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240bb2ee-e890-429a-8f16-a2e2d941a372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:03:01.853965 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240bb2ee-e890-429a-8f16-a2e2d941a372 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:01.854238 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240bb2ee-e890-429a-8f16-a2e2d941a372 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:01.854505 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240bb2ee-e890-429a-8f16-a2e2d941a372 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:01.861597 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240bb2ee-e890-429a-8f16-a2e2d941a372 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:03:01.863099 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240bb2ee-e890-429a-8f16-a2e2d941a372 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:03:01.874939 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-240bb2ee-e890-429a-8f16-a2e2d941a372 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 12 00:03:01.874939 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:03:01.874939 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:03:01.874939 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:03:01.874939 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:03:01.874939 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:03:01.874939 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:03:01.874939 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:03:01.874939 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:03:01.874939 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:03:01.874939 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:03:01.874939 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:03:01.874939 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 00:03:01.874939 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 00:03:01.874939 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 00:03:01.874939 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 00:03:01.880670 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:03:01.880670 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:03:01.880670 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 00:03:01.880670 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 00:03:01.880670 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 00:03:01.880670 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 00:03:01.880670 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:03:01.880670 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:03:01.880670 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:03:01.880670 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:03:01.880670 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:03:01.880670 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:03:01.880670 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:03:01.880670 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:03:01.880670 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:03:01.880670 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:03:01.880670 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:03:01.880670 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:03:01.881663 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:03:01.881663 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:03:01.881663 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 00:03:01.881663 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 00:03:01.881663 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 00:03:01.881663 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 00:03:01.881663 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 00:03:01.881663 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 00:03:01.881663 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 12 00:03:01.881663 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:03:01.881663 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 413/824] 10.222.0.36 () {56 vars in 1093 bytes} [Wed Oct 12 00:03:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:03:01.882254 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c366d28e-56dd-460a-b762-2b3410d5e468 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:03:01.960293 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:03:01.979237 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c366d28e-56dd-460a-b762-2b3410d5e468 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:03:02.013598 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b97a3b3-2989-422a-b086-ad50b354427d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:03:02.014192 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b97a3b3-2989-422a-b086-ad50b354427d None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:02.014465 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b97a3b3-2989-422a-b086-ad50b354427d None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:02.014729 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b97a3b3-2989-422a-b086-ad50b354427d None admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:02.015491 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b97a3b3-2989-422a-b086-ad50b354427d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:03:02.015789 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b97a3b3-2989-422a-b086-ad50b354427d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:03:02.017195 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b97a3b3-2989-422a-b086-ad50b354427d None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:03:02.034818 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 412/825] 10.222.0.36 () {56 vars in 1133 bytes} [Wed Oct 12 00:03:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 00:03:02.040545 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240bb2ee-e890-429a-8f16-a2e2d941a372 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:03:02.138958 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240bb2ee-e890-429a-8f16-a2e2d941a372 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:03:02.171926 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb351ab0-18af-48b1-ab84-baea3ae31783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:03:02.172529 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb351ab0-18af-48b1-ab84-baea3ae31783 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:02.172806 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb351ab0-18af-48b1-ab84-baea3ae31783 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:02.173073 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb351ab0-18af-48b1-ab84-baea3ae31783 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:02.206722 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:03:02.239317 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb351ab0-18af-48b1-ab84-baea3ae31783 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:03:02.240794 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb351ab0-18af-48b1-ab84-baea3ae31783 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:03:02.336922 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 414/826] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Oct 12 00:03:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 12 00:03:02.342675 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b97a3b3-2989-422a-b086-ad50b354427d None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:03:02.437934 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b97a3b3-2989-422a-b086-ad50b354427d None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:03:02.474266 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f55f9e-f56d-4cfd-9aa8-6e376d399774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:03:02.474916 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f55f9e-f56d-4cfd-9aa8-6e376d399774 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:02.475221 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f55f9e-f56d-4cfd-9aa8-6e376d399774 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:02.475533 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f55f9e-f56d-4cfd-9aa8-6e376d399774 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:02.482650 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:03:02.487169 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f55f9e-f56d-4cfd-9aa8-6e376d399774 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:03:02.488772 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f55f9e-f56d-4cfd-9aa8-6e376d399774 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:03:02.503045 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-47f55f9e-f56d-4cfd-9aa8-6e376d399774 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:03:02.503045 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:03:02.503045 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:03:02.503045 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:03:02.503045 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:03:02.503045 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:03:02.503045 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:03:02.503045 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:03:02.503045 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:03:02.503045 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:03:02.503045 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:03:02.503045 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:03:02.503045 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 00:03:02.503045 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 00:03:02.503045 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 00:03:02.503045 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 00:03:02.504193 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:03:02.504193 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:03:02.504193 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:03:02.504193 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:03:02.504193 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:03:02.504193 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:03:02.504193 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:03:02.504193 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:03:02.504193 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:03:02.504193 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:03:02.504193 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:03:02.504193 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:03:02.504193 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:03:02.504193 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:03:02.504193 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:03:02.504193 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:03:02.504193 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 00:03:02.505143 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 00:03:02.505143 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:03:02.505143 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:03:02.505143 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 413/827] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:03:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 00:03:02.510063 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb351ab0-18af-48b1-ab84-baea3ae31783 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:03:02.566689 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a5dbe1-459e-4fd5-8d43-97267428479b None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:02.566975 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a5dbe1-459e-4fd5-8d43-97267428479b None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:02.567246 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a5dbe1-459e-4fd5-8d43-97267428479b None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:02.568622 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 414/828] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:03:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 00:03:02.573605 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca761e8-6b7b-4889-8a51-373026a0b652 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:02.573882 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca761e8-6b7b-4889-8a51-373026a0b652 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:02.574151 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca761e8-6b7b-4889-8a51-373026a0b652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:02.580461 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:03:02.591856 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-1ca761e8-6b7b-4889-8a51-373026a0b652 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:03:02.593535 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-1ca761e8-6b7b-4889-8a51-373026a0b652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:03:02.615800 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb351ab0-18af-48b1-ab84-baea3ae31783 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:03:02.649047 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd27141-5855-4e5a-877d-709f76c5eaf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:03:02.649705 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd27141-5855-4e5a-877d-709f76c5eaf2 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:02.649980 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd27141-5855-4e5a-877d-709f76c5eaf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:02.650287 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd27141-5855-4e5a-877d-709f76c5eaf2 None admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:02.650932 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd27141-5855-4e5a-877d-709f76c5eaf2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:03:02.651234 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd27141-5855-4e5a-877d-709f76c5eaf2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:03:02.652627 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd27141-5855-4e5a-877d-709f76c5eaf2 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:03:02.658918 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 415/829] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:03:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 12 00:03:02.664545 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 415/830] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Oct 12 00:03:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:03:02.670016 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca761e8-6b7b-4889-8a51-373026a0b652 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:03:02.764908 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca761e8-6b7b-4889-8a51-373026a0b652 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:03:02.784217 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:03:02.798061 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe48842f-4715-483b-a5dc-95a401c66ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:03:02.798673 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe48842f-4715-483b-a5dc-95a401c66ce6 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:02.798952 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe48842f-4715-483b-a5dc-95a401c66ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:02.799224 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe48842f-4715-483b-a5dc-95a401c66ce6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:02.805754 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe48842f-4715-483b-a5dc-95a401c66ce6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:03:02.807184 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe48842f-4715-483b-a5dc-95a401c66ce6 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:03:02.818834 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-fe48842f-4715-483b-a5dc-95a401c66ce6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 00:03:02.818834 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:03:02.818834 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:03:02.818834 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:03:02.818834 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:03:02.818834 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:03:02.818834 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:03:02.818834 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:03:02.818834 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:03:02.818834 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:03:02.818834 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:03:02.818834 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:03:02.818834 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 00:03:02.818834 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 00:03:02.818834 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 00:03:02.819783 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 00:03:02.819783 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:03:02.819783 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:03:02.819783 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:03:02.819783 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:03:02.819783 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:03:02.819783 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:03:02.819783 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:03:02.819783 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:03:02.819783 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:03:02.819783 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:03:02.819783 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:03:02.819783 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:03:02.819783 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:03:02.819783 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:03:02.819783 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:03:02.819783 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:03:02.820983 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 00:03:02.820983 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 00:03:02.820983 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 00:03:02.820983 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 00:03:02.820983 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 00:03:02.820983 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 00:03:02.820983 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 00:03:02.820983 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:03:02.820983 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 416/831] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Oct 12 00:03:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 00:03:02.825651 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd27141-5855-4e5a-877d-709f76c5eaf2 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:03:02.922273 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd27141-5855-4e5a-877d-709f76c5eaf2 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:03:02.931701 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe48842f-4715-483b-a5dc-95a401c66ce6 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:03:02.948680 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:03:02.957000 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457c957f-f685-4644-adb8-29a3fee3dd0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:03:02.957613 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c957f-f685-4644-adb8-29a3fee3dd0a None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:02.957894 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c957f-f685-4644-adb8-29a3fee3dd0a None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:02.958167 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c957f-f685-4644-adb8-29a3fee3dd0a None admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:02.958852 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457c957f-f685-4644-adb8-29a3fee3dd0a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:03:02.959156 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457c957f-f685-4644-adb8-29a3fee3dd0a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:03:02.960644 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457c957f-f685-4644-adb8-29a3fee3dd0a None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:03:02.971763 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 416/832] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Oct 12 00:03:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:03:02.977655 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457c957f-f685-4644-adb8-29a3fee3dd0a None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:03:03.030224 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe48842f-4715-483b-a5dc-95a401c66ce6 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:03:03.062171 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d99b64-da9e-448a-83c3-95c70b6dbe0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:03:03.062788 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d99b64-da9e-448a-83c3-95c70b6dbe0c None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:03.063059 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d99b64-da9e-448a-83c3-95c70b6dbe0c None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:03.063323 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d99b64-da9e-448a-83c3-95c70b6dbe0c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:03.073865 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d99b64-da9e-448a-83c3-95c70b6dbe0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 00:03:03.076041 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457c957f-f685-4644-adb8-29a3fee3dd0a None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:03:03.080039 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-d1d99b64-da9e-448a-83c3-95c70b6dbe0c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 00:03:03.080039 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:03:03.080039 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:03:03.080039 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:03:03.080039 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:03:03.080039 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:03:03.080039 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:03:03.080039 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:03:03.080039 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:03:03.080039 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:03:03.080039 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:03:03.080039 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:03:03.080039 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 00:03:03.080039 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 00:03:03.080039 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 00:03:03.080039 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 00:03:03.080967 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 00:03:03.080967 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 00:03:03.080967 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 00:03:03.080967 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 00:03:03.080967 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:03:03.080967 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:03:03.080967 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:03:03.080967 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:03:03.080967 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:03:03.080967 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:03:03.080967 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:03:03.080967 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:03:03.080967 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:03:03.080967 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:03:03.080967 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:03:03.080967 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:03:03.081848 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:03:03.081848 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:03:03.081848 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:03:03.081848 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:03:03.081848 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 00:03:03.081848 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 00:03:03.081848 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 00:03:03.081848 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 00:03:03.081848 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 00:03:03.081848 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 00:03:03.081848 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 00:03:03.081848 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:03:03.083514 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 417/833] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Oct 12 00:03:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 00:03:03.086950 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d99b64-da9e-448a-83c3-95c70b6dbe0c None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:03:03.110579 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea9e055-fb9b-432c-9e9e-95a161cd7fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:03:03.111178 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea9e055-fb9b-432c-9e9e-95a161cd7fc7 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:03.111494 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea9e055-fb9b-432c-9e9e-95a161cd7fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:03.111760 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea9e055-fb9b-432c-9e9e-95a161cd7fc7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:03.112471 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea9e055-fb9b-432c-9e9e-95a161cd7fc7 None admin] RBAC: Adding query filter params (role.id=59b6984c4af34bffa59fdac6d215a6a3, scope.project.id=0697927c6aac43d4b14a750fbbb1145e, user.id=a515b75aa3944912ad2364ffd7924f40) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:03:03.112770 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea9e055-fb9b-432c-9e9e-95a161cd7fc7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:03:03.114264 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea9e055-fb9b-432c-9e9e-95a161cd7fc7 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:03:03.120025 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 417/834] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Oct 12 00:03:02 2022] GET /identity/v3/role_assignments?user.id=a515b75aa3944912ad2364ffd7924f40&scope.project.id=0697927c6aac43d4b14a750fbbb1145e&role.id=59b6984c4af34bffa59fdac6d215a6a3 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 00:03:03.128468 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:03:03.189707 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d99b64-da9e-448a-83c3-95c70b6dbe0c None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:03:03.225532 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f7fbb5-9413-4a1c-b7fd-940e68f1f3cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:03:03.226151 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f7fbb5-9413-4a1c-b7fd-940e68f1f3cb None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:03.226428 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f7fbb5-9413-4a1c-b7fd-940e68f1f3cb None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:03.226697 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f7fbb5-9413-4a1c-b7fd-940e68f1f3cb None admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:03.227360 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f7fbb5-9413-4a1c-b7fd-940e68f1f3cb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:03:03.227680 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f7fbb5-9413-4a1c-b7fd-940e68f1f3cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:03:03.229056 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f7fbb5-9413-4a1c-b7fd-940e68f1f3cb None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:03:03.237966 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 418/835] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Oct 12 00:03:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 00:03:03.244289 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea9e055-fb9b-432c-9e9e-95a161cd7fc7 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:03:03.317103 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:03:03.363692 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea9e055-fb9b-432c-9e9e-95a161cd7fc7 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:03:03.397557 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009af671-ae63-4f8a-9935-bc377ec82248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:03:03.398187 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009af671-ae63-4f8a-9935-bc377ec82248 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:03.398463 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009af671-ae63-4f8a-9935-bc377ec82248 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:03.398729 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009af671-ae63-4f8a-9935-bc377ec82248 None admin] PATH_INFO: `/v3/users/010943d8107746cea1ced6743918667a` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:03.399877 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009af671-ae63-4f8a-9935-bc377ec82248 None admin] RBAC: Authorizing `identity:get_user(user_id=010943d8107746cea1ced6743918667a)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:03:03.401302 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009af671-ae63-4f8a-9935-bc377ec82248 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:03:03.402905 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 418/836] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Oct 12 00:03:03 2022] GET /identity/v3/users/010943d8107746cea1ced6743918667a => generated 285 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:03:03.409131 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f7fbb5-9413-4a1c-b7fd-940e68f1f3cb None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:03:03.510014 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f7fbb5-9413-4a1c-b7fd-940e68f1f3cb None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:03:03.511599 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:03:03.546597 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2160ae80-527f-4fd2-b203-1f332a1281c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:03:03.546792 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2160ae80-527f-4fd2-b203-1f332a1281c1 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:03.547069 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2160ae80-527f-4fd2-b203-1f332a1281c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:03.547339 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2160ae80-527f-4fd2-b203-1f332a1281c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:03.616221 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2160ae80-527f-4fd2-b203-1f332a1281c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:03:03.617676 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2160ae80-527f-4fd2-b203-1f332a1281c1 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:03:03.715623 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 419/837] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Oct 12 00:03:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 12 00:03:03.721869 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009af671-ae63-4f8a-9935-bc377ec82248 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:03:03.830381 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009af671-ae63-4f8a-9935-bc377ec82248 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:03:03.852055 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:03:03.867061 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b65d5c-bc3b-4b6a-867e-d6c81e3e1e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:03:03.867662 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b65d5c-bc3b-4b6a-867e-d6c81e3e1e44 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:03.867940 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b65d5c-bc3b-4b6a-867e-d6c81e3e1e44 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:03.868215 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b65d5c-bc3b-4b6a-867e-d6c81e3e1e44 None admin] PATH_INFO: `/v3/projects/8aea0d97fb5a489a9e29c789b91976f1` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:03.869358 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b65d5c-bc3b-4b6a-867e-d6c81e3e1e44 None admin] RBAC: Authorizing `identity:get_project(project_id=8aea0d97fb5a489a9e29c789b91976f1)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:03:03.870781 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b65d5c-bc3b-4b6a-867e-d6c81e3e1e44 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:03:03.872560 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 419/838] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Oct 12 00:03:03 2022] GET /identity/v3/projects/8aea0d97fb5a489a9e29c789b91976f1 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:03:03.879716 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2160ae80-527f-4fd2-b203-1f332a1281c1 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:03:03.881878 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:03:03.980738 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2160ae80-527f-4fd2-b203-1f332a1281c1 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:03:04.014306 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d707b4f-972f-4162-8d74-d0b923759f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:03:04.014932 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d707b4f-972f-4162-8d74-d0b923759f6c None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:04.015205 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d707b4f-972f-4162-8d74-d0b923759f6c None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:04.015504 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d707b4f-972f-4162-8d74-d0b923759f6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:04.016214 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d707b4f-972f-4162-8d74-d0b923759f6c None admin] RBAC: Adding query filter params (role.id=b2e41f0d7a304dc7af8b6a232049b6a2, scope.project.id=8aea0d97fb5a489a9e29c789b91976f1, user.id=010943d8107746cea1ced6743918667a) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:03:04.016514 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d707b4f-972f-4162-8d74-d0b923759f6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:03:04.017921 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d707b4f-972f-4162-8d74-d0b923759f6c None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:03:04.023633 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 420/839] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Oct 12 00:03:03 2022] GET /identity/v3/role_assignments?user.id=010943d8107746cea1ced6743918667a&scope.project.id=8aea0d97fb5a489a9e29c789b91976f1&role.id=b2e41f0d7a304dc7af8b6a232049b6a2 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:03:06.163901 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3482cf31-4f2c-48f5-88d4-af607a7b5971 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:06.164453 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3482cf31-4f2c-48f5-88d4-af607a7b5971 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:06.164453 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3482cf31-4f2c-48f5-88d4-af607a7b5971 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:06.165786 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 420/840] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:03:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:03:06.171564 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4545699b-9410-4d29-9550-7d4719648b63 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:06.171950 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4545699b-9410-4d29-9550-7d4719648b63 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:06.172108 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4545699b-9410-4d29-9550-7d4719648b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:06.190368 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-4545699b-9410-4d29-9550-7d4719648b63 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:03:06.191613 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-4545699b-9410-4d29-9550-7d4719648b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:03:06.248789 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:03:06.255990 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 421/841] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:03:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 00:03:06.522511 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3482cf31-4f2c-48f5-88d4-af607a7b5971 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:03:06.620772 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3482cf31-4f2c-48f5-88d4-af607a7b5971 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:03:06.653957 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f49f62c-d71b-4438-8b5c-adc49c436f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:03:06.654571 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f49f62c-d71b-4438-8b5c-adc49c436f13 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:06.654842 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f49f62c-d71b-4438-8b5c-adc49c436f13 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:06.655687 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f49f62c-d71b-4438-8b5c-adc49c436f13 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:06.656788 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f49f62c-d71b-4438-8b5c-adc49c436f13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:03:06.658203 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f49f62c-d71b-4438-8b5c-adc49c436f13 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:03:06.659765 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 421/842] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:03:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:03:06.665416 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4545699b-9410-4d29-9550-7d4719648b63 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:03:06.759753 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4545699b-9410-4d29-9550-7d4719648b63 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:03:06.792211 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e197e0a1-7a53-4428-834c-b7c172becf3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:03:06.792745 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e197e0a1-7a53-4428-834c-b7c172becf3f None admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:06.793016 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e197e0a1-7a53-4428-834c-b7c172becf3f None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:06.793430 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e197e0a1-7a53-4428-834c-b7c172becf3f None admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:06.794067 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e197e0a1-7a53-4428-834c-b7c172becf3f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:03:06.795542 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e197e0a1-7a53-4428-834c-b7c172becf3f None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:03:06.895122 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 422/843] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Oct 12 00:03:06 2022] POST /identity/v3/users => generated 286 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:03:09.022227 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b2a669-7bbb-4b2b-9c39-ba1fa578f8c0 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:09.022761 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b2a669-7bbb-4b2b-9c39-ba1fa578f8c0 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:09.022761 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b2a669-7bbb-4b2b-9c39-ba1fa578f8c0 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:09.024144 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 422/844] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:03:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:03:09.029382 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acc5f57-ced3-4821-92ee-6f2e9504f86f None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:09.029811 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acc5f57-ced3-4821-92ee-6f2e9504f86f None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:09.030596 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acc5f57-ced3-4821-92ee-6f2e9504f86f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:09.048312 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-5acc5f57-ced3-4821-92ee-6f2e9504f86f None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:03:09.049495 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-5acc5f57-ced3-4821-92ee-6f2e9504f86f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:03:09.094781 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:03:09.113361 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 423/845] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:03:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 00:03:09.384486 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b2a669-7bbb-4b2b-9c39-ba1fa578f8c0 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:03:09.439680 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:03:09.482414 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b2a669-7bbb-4b2b-9c39-ba1fa578f8c0 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:03:09.515210 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c003400f-e56a-4291-bc92-0dd9c4fc0f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:03:09.515852 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c003400f-e56a-4291-bc92-0dd9c4fc0f22 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:09.516130 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c003400f-e56a-4291-bc92-0dd9c4fc0f22 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:09.516399 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c003400f-e56a-4291-bc92-0dd9c4fc0f22 None admin] PATH_INFO: `/v3/roles/81991615ac0b46818426bf37015df930` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:09.517784 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c003400f-e56a-4291-bc92-0dd9c4fc0f22 None admin] RBAC: Authorizing `identity:get_role(role_id=81991615ac0b46818426bf37015df930)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:03:09.519146 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c003400f-e56a-4291-bc92-0dd9c4fc0f22 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:03:09.520472 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 423/846] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Oct 12 00:03:09 2022] GET /identity/v3/roles/81991615ac0b46818426bf37015df930 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:03:09.526312 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acc5f57-ced3-4821-92ee-6f2e9504f86f None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:03:09.620185 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acc5f57-ced3-4821-92ee-6f2e9504f86f None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:03:09.652645 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b694a2-0027-4c4a-b339-d2dc66da391e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:03:09.653207 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b694a2-0027-4c4a-b339-d2dc66da391e None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:09.653471 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b694a2-0027-4c4a-b339-d2dc66da391e None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:09.653741 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b694a2-0027-4c4a-b339-d2dc66da391e None admin] PATH_INFO: `/v3/users/64ea6659aba64c1581b9295736532172` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:09.666454 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b694a2-0027-4c4a-b339-d2dc66da391e None admin] RBAC: Authorizing `identity:get_user(user_id=64ea6659aba64c1581b9295736532172)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:03:09.667927 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b694a2-0027-4c4a-b339-d2dc66da391e None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:03:09.669547 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 424/847] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Oct 12 00:03:09 2022] GET /identity/v3/users/64ea6659aba64c1581b9295736532172 => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:03:09.675762 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c003400f-e56a-4291-bc92-0dd9c4fc0f22 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:03:09.771740 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c003400f-e56a-4291-bc92-0dd9c4fc0f22 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:03:09.793506 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:03:09.804149 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a58db6-13a3-4f1f-a954-0b733225563a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:03:09.821050 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a58db6-13a3-4f1f-a954-0b733225563a None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:09.821050 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a58db6-13a3-4f1f-a954-0b733225563a None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:09.821050 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a58db6-13a3-4f1f-a954-0b733225563a None admin] PATH_INFO: `/v3/projects/8aea0d97fb5a489a9e29c789b91976f1` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:09.821050 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a58db6-13a3-4f1f-a954-0b733225563a None admin] RBAC: Authorizing `identity:get_project(project_id=8aea0d97fb5a489a9e29c789b91976f1)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:03:09.821050 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a58db6-13a3-4f1f-a954-0b733225563a None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:03:09.821050 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 424/848] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Oct 12 00:03:09 2022] GET /identity/v3/projects/8aea0d97fb5a489a9e29c789b91976f1 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:03:09.821547 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b694a2-0027-4c4a-b339-d2dc66da391e None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:03:09.855945 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:03:09.913355 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b694a2-0027-4c4a-b339-d2dc66da391e None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:03:09.946637 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119b5291-fb29-46d7-94c3-5128f8a07bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:03:09.947280 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119b5291-fb29-46d7-94c3-5128f8a07bdd None admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:09.947586 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119b5291-fb29-46d7-94c3-5128f8a07bdd None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:09.947862 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119b5291-fb29-46d7-94c3-5128f8a07bdd None admin] PATH_INFO: `/v3/projects/8aea0d97fb5a489a9e29c789b91976f1/users/64ea6659aba64c1581b9295736532172/roles/81991615ac0b46818426bf37015df930` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:09.949988 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119b5291-fb29-46d7-94c3-5128f8a07bdd None admin] RBAC: Authorizing `identity:create_grant(project_id=8aea0d97fb5a489a9e29c789b91976f1, user_id=64ea6659aba64c1581b9295736532172, role_id=81991615ac0b46818426bf37015df930)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:03:09.951439 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119b5291-fb29-46d7-94c3-5128f8a07bdd None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:03:10.060477 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 425/849] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Oct 12 00:03:09 2022] PUT /identity/v3/projects/8aea0d97fb5a489a9e29c789b91976f1/users/64ea6659aba64c1581b9295736532172/roles/81991615ac0b46818426bf37015df930 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:03:12.159104 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c05166-16af-4d67-b146-e79b4c37d05c None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:12.159681 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c05166-16af-4d67-b146-e79b4c37d05c None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:12.159681 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c05166-16af-4d67-b146-e79b4c37d05c None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:12.161004 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 425/850] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:03:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:03:12.166412 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da14f49f-7900-4758-87ea-2bbefd2bccd6 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:12.166829 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da14f49f-7900-4758-87ea-2bbefd2bccd6 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:12.166966 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da14f49f-7900-4758-87ea-2bbefd2bccd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:12.185308 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-da14f49f-7900-4758-87ea-2bbefd2bccd6 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:03:12.187106 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-da14f49f-7900-4758-87ea-2bbefd2bccd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:03:12.251792 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 426/851] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:03:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 00:03:12.516939 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c05166-16af-4d67-b146-e79b4c37d05c None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:03:12.615868 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c05166-16af-4d67-b146-e79b4c37d05c None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:03:12.648319 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec31a75a-d166-4cae-b71f-52259225f0fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:03:12.648935 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec31a75a-d166-4cae-b71f-52259225f0fc None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:12.649210 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec31a75a-d166-4cae-b71f-52259225f0fc None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:12.649478 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec31a75a-d166-4cae-b71f-52259225f0fc None admin] PATH_INFO: `/v3/roles/81991615ac0b46818426bf37015df930` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:12.650836 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec31a75a-d166-4cae-b71f-52259225f0fc None admin] RBAC: Authorizing `identity:get_role(role_id=81991615ac0b46818426bf37015df930)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:03:12.652216 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec31a75a-d166-4cae-b71f-52259225f0fc None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:03:12.653502 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 426/852] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Oct 12 00:03:12 2022] GET /identity/v3/roles/81991615ac0b46818426bf37015df930 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:03:12.659863 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da14f49f-7900-4758-87ea-2bbefd2bccd6 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:03:12.687788 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:03:12.754169 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da14f49f-7900-4758-87ea-2bbefd2bccd6 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:03:12.788761 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8fc475-391f-4e48-86b9-174a9c17c654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:03:12.789324 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8fc475-391f-4e48-86b9-174a9c17c654 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:12.789602 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8fc475-391f-4e48-86b9-174a9c17c654 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:12.789874 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8fc475-391f-4e48-86b9-174a9c17c654 None admin] PATH_INFO: `/v3/users/64ea6659aba64c1581b9295736532172` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:12.791066 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8fc475-391f-4e48-86b9-174a9c17c654 None admin] RBAC: Authorizing `identity:get_user(user_id=64ea6659aba64c1581b9295736532172)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:03:12.792522 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8fc475-391f-4e48-86b9-174a9c17c654 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:03:12.794083 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 427/853] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Oct 12 00:03:12 2022] GET /identity/v3/users/64ea6659aba64c1581b9295736532172 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:03:12.799784 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec31a75a-d166-4cae-b71f-52259225f0fc None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:03:12.856231 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:03:12.895279 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec31a75a-d166-4cae-b71f-52259225f0fc None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:03:12.926861 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c95fe9-cbf6-466a-b32b-4c315790e380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:03:12.927478 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c95fe9-cbf6-466a-b32b-4c315790e380 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:12.927755 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c95fe9-cbf6-466a-b32b-4c315790e380 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:12.928022 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c95fe9-cbf6-466a-b32b-4c315790e380 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:12.990760 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c95fe9-cbf6-466a-b32b-4c315790e380 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:03:12.992204 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c95fe9-cbf6-466a-b32b-4c315790e380 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:03:13.046728 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:03:13.087188 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 427/854] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Oct 12 00:03:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 12 00:03:13.093328 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8fc475-391f-4e48-86b9-174a9c17c654 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:03:13.183861 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:03:13.188032 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8fc475-391f-4e48-86b9-174a9c17c654 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:03:13.221545 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213dceff-08c3-4bcc-a440-40885d979971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:03:13.222151 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213dceff-08c3-4bcc-a440-40885d979971 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:13.222425 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213dceff-08c3-4bcc-a440-40885d979971 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:13.222692 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213dceff-08c3-4bcc-a440-40885d979971 None admin] PATH_INFO: `/v3/projects/8aea0d97fb5a489a9e29c789b91976f1` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:13.223807 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213dceff-08c3-4bcc-a440-40885d979971 None admin] RBAC: Authorizing `identity:get_project(project_id=8aea0d97fb5a489a9e29c789b91976f1)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:03:13.225211 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213dceff-08c3-4bcc-a440-40885d979971 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:03:13.226784 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 428/855] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Oct 12 00:03:13 2022] GET /identity/v3/projects/8aea0d97fb5a489a9e29c789b91976f1 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:03:13.233066 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c95fe9-cbf6-466a-b32b-4c315790e380 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:03:13.328480 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c95fe9-cbf6-466a-b32b-4c315790e380 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:03:13.360827 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76496af-9c70-4d7a-8779-d8435e7eb15a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:03:13.361412 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76496af-9c70-4d7a-8779-d8435e7eb15a None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:13.361688 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76496af-9c70-4d7a-8779-d8435e7eb15a None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:13.361958 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76496af-9c70-4d7a-8779-d8435e7eb15a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:13.362668 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76496af-9c70-4d7a-8779-d8435e7eb15a None admin] RBAC: Adding query filter params (role.id=81991615ac0b46818426bf37015df930, scope.project.id=8aea0d97fb5a489a9e29c789b91976f1, user.id=64ea6659aba64c1581b9295736532172) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:03:13.362971 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76496af-9c70-4d7a-8779-d8435e7eb15a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:03:13.364413 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76496af-9c70-4d7a-8779-d8435e7eb15a None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:03:13.369855 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 428/856] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Oct 12 00:03:13 2022] GET /identity/v3/role_assignments?user.id=64ea6659aba64c1581b9295736532172&scope.project.id=8aea0d97fb5a489a9e29c789b91976f1&role.id=81991615ac0b46818426bf37015df930 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:03:15.475765 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5e17f-678d-4fe2-834a-85215fe0a13c None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:15.476296 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5e17f-678d-4fe2-834a-85215fe0a13c None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:15.476381 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5e17f-678d-4fe2-834a-85215fe0a13c None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:15.477666 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 429/857] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:03:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:03:15.483705 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665cba1e-2263-4737-97bf-73a3488d33c8 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:15.484137 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665cba1e-2263-4737-97bf-73a3488d33c8 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:15.484243 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665cba1e-2263-4737-97bf-73a3488d33c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:15.501836 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-665cba1e-2263-4737-97bf-73a3488d33c8 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:03:15.503003 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-665cba1e-2263-4737-97bf-73a3488d33c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:03:15.519887 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:03:15.568328 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 429/858] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:03:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 00:03:15.838069 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b5e17f-678d-4fe2-834a-85215fe0a13c None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:03:15.938034 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b5e17f-678d-4fe2-834a-85215fe0a13c None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:03:15.970625 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab395d0-b61c-4af2-9119-3a33163c5b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:03:15.971237 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab395d0-b61c-4af2-9119-3a33163c5b73 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:15.971543 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab395d0-b61c-4af2-9119-3a33163c5b73 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:15.971817 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab395d0-b61c-4af2-9119-3a33163c5b73 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:15.972918 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab395d0-b61c-4af2-9119-3a33163c5b73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:03:15.974361 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab395d0-b61c-4af2-9119-3a33163c5b73 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:03:15.975912 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 430/859] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:03:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:03:15.981490 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665cba1e-2263-4737-97bf-73a3488d33c8 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:03:16.075468 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665cba1e-2263-4737-97bf-73a3488d33c8 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:03:16.107695 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ad146b-a5a3-4828-8e3f-b46883c81474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:03:16.108223 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad146b-a5a3-4828-8e3f-b46883c81474 None admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:16.108496 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad146b-a5a3-4828-8e3f-b46883c81474 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:16.108762 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad146b-a5a3-4828-8e3f-b46883c81474 None admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:16.109393 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ad146b-a5a3-4828-8e3f-b46883c81474 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:03:16.110808 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ad146b-a5a3-4828-8e3f-b46883c81474 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:03:16.193270 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 430/860] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Oct 12 00:03:15 2022] POST /identity/v3/projects => generated 311 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:03:18.323446 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a480e6-7897-4afc-802d-0868fb3b7b75 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:18.323979 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a480e6-7897-4afc-802d-0868fb3b7b75 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:18.323979 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a480e6-7897-4afc-802d-0868fb3b7b75 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:18.325294 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 431/861] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:03:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:03:18.330527 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c13537-7100-43c5-a891-f0c34d433435 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:18.330945 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c13537-7100-43c5-a891-f0c34d433435 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:18.331059 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c13537-7100-43c5-a891-f0c34d433435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:18.344403 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:03:18.349619 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-07c13537-7100-43c5-a891-f0c34d433435 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:03:18.350799 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-07c13537-7100-43c5-a891-f0c34d433435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:03:18.415344 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 431/862] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:03:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 00:03:18.680748 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a480e6-7897-4afc-802d-0868fb3b7b75 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:03:18.714535 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:03:18.778457 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a480e6-7897-4afc-802d-0868fb3b7b75 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:03:18.810725 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efafeb0-863d-4396-b4a6-5cd377e103cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:03:18.811336 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efafeb0-863d-4396-b4a6-5cd377e103cc None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:18.811652 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efafeb0-863d-4396-b4a6-5cd377e103cc None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:18.811945 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efafeb0-863d-4396-b4a6-5cd377e103cc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:18.813041 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efafeb0-863d-4396-b4a6-5cd377e103cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:03:18.814460 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efafeb0-863d-4396-b4a6-5cd377e103cc None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:03:18.816017 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 432/863] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:03:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:03:18.821541 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c13537-7100-43c5-a891-f0c34d433435 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:03:18.916478 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c13537-7100-43c5-a891-f0c34d433435 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:03:18.948731 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273aafdb-0880-4b4c-9680-427fad5169e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:03:18.949268 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273aafdb-0880-4b4c-9680-427fad5169e8 None admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:18.949545 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273aafdb-0880-4b4c-9680-427fad5169e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:18.949814 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273aafdb-0880-4b4c-9680-427fad5169e8 None admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:18.950463 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273aafdb-0880-4b4c-9680-427fad5169e8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:03:18.952503 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273aafdb-0880-4b4c-9680-427fad5169e8 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:03:18.961127 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:03:19.054337 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 432/864] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Oct 12 00:03:18 2022] POST /identity/v3/users => generated 286 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:03:21.197598 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f18b40d-3e6d-43dd-a137-203dbee0327f None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:21.198166 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f18b40d-3e6d-43dd-a137-203dbee0327f None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:21.198166 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f18b40d-3e6d-43dd-a137-203dbee0327f None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:21.199473 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 433/865] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:03:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:03:21.204648 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fe152e-1469-48c4-a112-4c37b959da83 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:21.205063 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fe152e-1469-48c4-a112-4c37b959da83 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:21.205191 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fe152e-1469-48c4-a112-4c37b959da83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:21.223293 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-10fe152e-1469-48c4-a112-4c37b959da83 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:03:21.226033 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-10fe152e-1469-48c4-a112-4c37b959da83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:03:21.288849 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 433/866] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:03:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 00:03:21.556307 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f18b40d-3e6d-43dd-a137-203dbee0327f None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:03:21.653978 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f18b40d-3e6d-43dd-a137-203dbee0327f None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:03:21.656021 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:03:21.686360 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02291ccc-6847-4b35-b4f7-ded3827d4a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:03:21.686976 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02291ccc-6847-4b35-b4f7-ded3827d4a18 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:21.687258 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02291ccc-6847-4b35-b4f7-ded3827d4a18 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:21.687555 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02291ccc-6847-4b35-b4f7-ded3827d4a18 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:21.697253 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02291ccc-6847-4b35-b4f7-ded3827d4a18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 00:03:21.702627 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-02291ccc-6847-4b35-b4f7-ded3827d4a18 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 00:03:21.702627 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:03:21.702627 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:03:21.702627 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:03:21.702627 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:03:21.702627 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:03:21.702627 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:03:21.702627 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:03:21.702627 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:03:21.702627 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:03:21.702627 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:03:21.702627 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:03:21.702627 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 00:03:21.702627 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 00:03:21.702627 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 00:03:21.702627 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 00:03:21.703816 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 00:03:21.703816 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 00:03:21.703816 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 00:03:21.703816 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 00:03:21.703816 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:03:21.703816 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:03:21.703816 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:03:21.703816 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:03:21.703816 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:03:21.703816 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:03:21.703816 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:03:21.703816 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:03:21.703816 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:03:21.703816 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:03:21.703816 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:03:21.703816 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:03:21.704695 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:03:21.704695 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:03:21.704695 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:03:21.704695 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:03:21.704695 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 00:03:21.704695 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 00:03:21.704695 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 00:03:21.704695 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 00:03:21.704695 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 00:03:21.704695 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 00:03:21.704695 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 00:03:21.704695 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:03:21.704695 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 434/867] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Oct 12 00:03:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:03:21.709691 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fe152e-1469-48c4-a112-4c37b959da83 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:03:21.804453 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fe152e-1469-48c4-a112-4c37b959da83 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:03:21.825328 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:03:21.836683 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4608135-4d49-487f-8aab-2f100e4ac8c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:03:21.837274 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4608135-4d49-487f-8aab-2f100e4ac8c2 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:21.837546 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4608135-4d49-487f-8aab-2f100e4ac8c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:21.837814 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4608135-4d49-487f-8aab-2f100e4ac8c2 None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:21.838464 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4608135-4d49-487f-8aab-2f100e4ac8c2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:03:21.838760 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4608135-4d49-487f-8aab-2f100e4ac8c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:03:21.840141 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4608135-4d49-487f-8aab-2f100e4ac8c2 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:03:21.848326 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 434/868] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Oct 12 00:03:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:03:21.854027 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02291ccc-6847-4b35-b4f7-ded3827d4a18 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:03:21.949134 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02291ccc-6847-4b35-b4f7-ded3827d4a18 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:03:21.981308 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9adf69-3252-40b9-bd99-452b2475b583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:03:21.981924 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9adf69-3252-40b9-bd99-452b2475b583 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:21.982225 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9adf69-3252-40b9-bd99-452b2475b583 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:21.982495 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9adf69-3252-40b9-bd99-452b2475b583 None admin] PATH_INFO: `/v3/users/d6b7beee0c7c40198e7e1403feaa7cf5` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:21.995407 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9adf69-3252-40b9-bd99-452b2475b583 None admin] RBAC: Authorizing `identity:get_user(user_id=d6b7beee0c7c40198e7e1403feaa7cf5)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:03:21.996824 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9adf69-3252-40b9-bd99-452b2475b583 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:03:21.998880 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 435/869] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Oct 12 00:03:21 2022] GET /identity/v3/users/d6b7beee0c7c40198e7e1403feaa7cf5 => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:03:22.004667 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4608135-4d49-487f-8aab-2f100e4ac8c2 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:03:22.100617 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4608135-4d49-487f-8aab-2f100e4ac8c2 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:03:22.132581 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc426aa-aaf7-43ad-a3fc-ae5a09481a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:03:22.133197 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc426aa-aaf7-43ad-a3fc-ae5a09481a93 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:22.133469 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc426aa-aaf7-43ad-a3fc-ae5a09481a93 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:22.133735 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc426aa-aaf7-43ad-a3fc-ae5a09481a93 None admin] PATH_INFO: `/v3/projects/25a0af5a7de34484a634ceab3ae562e8` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:22.134832 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc426aa-aaf7-43ad-a3fc-ae5a09481a93 None admin] RBAC: Authorizing `identity:get_project(project_id=25a0af5a7de34484a634ceab3ae562e8)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:03:22.136275 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc426aa-aaf7-43ad-a3fc-ae5a09481a93 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:03:22.137849 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 435/870] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Oct 12 00:03:22 2022] GET /identity/v3/projects/25a0af5a7de34484a634ceab3ae562e8 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:03:22.144253 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9adf69-3252-40b9-bd99-452b2475b583 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:03:22.164699 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:03:22.237642 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9adf69-3252-40b9-bd99-452b2475b583 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:03:22.270801 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcec9e9e-bb22-4e37-b145-c91b7c7ee51e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:03:22.271464 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcec9e9e-bb22-4e37-b145-c91b7c7ee51e None admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:22.271741 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcec9e9e-bb22-4e37-b145-c91b7c7ee51e None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:22.272007 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcec9e9e-bb22-4e37-b145-c91b7c7ee51e None admin] PATH_INFO: `/v3/projects/25a0af5a7de34484a634ceab3ae562e8/users/d6b7beee0c7c40198e7e1403feaa7cf5/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:22.273873 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcec9e9e-bb22-4e37-b145-c91b7c7ee51e None admin] RBAC: Authorizing `identity:create_grant(project_id=25a0af5a7de34484a634ceab3ae562e8, user_id=d6b7beee0c7c40198e7e1403feaa7cf5, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:03:22.275284 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcec9e9e-bb22-4e37-b145-c91b7c7ee51e None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:03:22.357639 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 436/871] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Oct 12 00:03:22 2022] PUT /identity/v3/projects/25a0af5a7de34484a634ceab3ae562e8/users/d6b7beee0c7c40198e7e1403feaa7cf5/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:03:24.472719 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1837bcb2-ed67-4225-bfc0-cfbdb202c38a None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:24.473248 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1837bcb2-ed67-4225-bfc0-cfbdb202c38a None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:24.473248 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1837bcb2-ed67-4225-bfc0-cfbdb202c38a None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:24.474609 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 436/872] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:03:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:03:24.479890 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1342ae-d1e6-46c2-8065-bfc8f517dbd0 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:24.480306 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1342ae-d1e6-46c2-8065-bfc8f517dbd0 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:24.480433 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1342ae-d1e6-46c2-8065-bfc8f517dbd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:24.498523 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-1e1342ae-d1e6-46c2-8065-bfc8f517dbd0 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:03:24.499740 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-1e1342ae-d1e6-46c2-8065-bfc8f517dbd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:03:24.564712 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 437/873] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:03:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 00:03:24.831606 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1837bcb2-ed67-4225-bfc0-cfbdb202c38a None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:03:24.883723 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:03:24.928167 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1837bcb2-ed67-4225-bfc0-cfbdb202c38a None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:03:24.960388 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e65dd7e-d09b-47b4-ade3-10490868a247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:03:24.961000 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e65dd7e-d09b-47b4-ade3-10490868a247 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:24.961275 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e65dd7e-d09b-47b4-ade3-10490868a247 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:24.961544 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e65dd7e-d09b-47b4-ade3-10490868a247 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:24.972326 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e65dd7e-d09b-47b4-ade3-10490868a247 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 00:03:24.977104 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-4e65dd7e-d09b-47b4-ade3-10490868a247 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 00:03:24.977104 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:03:24.977104 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:03:24.977104 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:03:24.977104 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:03:24.977104 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:03:24.977104 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:03:24.977104 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:03:24.977104 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:03:24.977104 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:03:24.977104 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:03:24.977104 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:03:24.977104 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 00:03:24.977104 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 00:03:24.977104 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 00:03:24.977104 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 00:03:24.978184 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 00:03:24.978184 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 00:03:24.978184 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 00:03:24.978184 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 00:03:24.978184 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:03:24.978184 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:03:24.978184 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:03:24.978184 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:03:24.978184 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:03:24.978184 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:03:24.978184 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:03:24.978184 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:03:24.978184 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:03:24.978184 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:03:24.978184 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:03:24.978184 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:03:24.979237 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:03:24.979237 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:03:24.979237 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:03:24.979237 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:03:24.979237 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 00:03:24.979237 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 00:03:24.979237 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 00:03:24.979237 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 00:03:24.979237 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 00:03:24.979237 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 00:03:24.979237 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 00:03:24.979237 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:03:24.979237 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 437/874] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Oct 12 00:03:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:03:24.984768 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1342ae-d1e6-46c2-8065-bfc8f517dbd0 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:03:24.996009 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:03:25.078094 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1342ae-d1e6-46c2-8065-bfc8f517dbd0 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:03:25.110589 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18443721-4040-432c-b49e-9c99e29af0c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:03:25.111126 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18443721-4040-432c-b49e-9c99e29af0c1 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:25.111420 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18443721-4040-432c-b49e-9c99e29af0c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:25.111690 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18443721-4040-432c-b49e-9c99e29af0c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:25.112423 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18443721-4040-432c-b49e-9c99e29af0c1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:03:25.112721 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18443721-4040-432c-b49e-9c99e29af0c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:03:25.114067 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18443721-4040-432c-b49e-9c99e29af0c1 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:03:25.121641 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 438/875] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Oct 12 00:03:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:03:25.127398 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e65dd7e-d09b-47b4-ade3-10490868a247 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:03:25.223260 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e65dd7e-d09b-47b4-ade3-10490868a247 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:03:25.235949 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:03:25.255604 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c712a53a-848d-4a38-b17d-a72c264cab38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:03:25.256222 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c712a53a-848d-4a38-b17d-a72c264cab38 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:25.256501 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c712a53a-848d-4a38-b17d-a72c264cab38 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:25.256771 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c712a53a-848d-4a38-b17d-a72c264cab38 None admin] PATH_INFO: `/v3/users/d6b7beee0c7c40198e7e1403feaa7cf5` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:25.258135 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c712a53a-848d-4a38-b17d-a72c264cab38 None admin] RBAC: Authorizing `identity:get_user(user_id=d6b7beee0c7c40198e7e1403feaa7cf5)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:03:25.259589 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c712a53a-848d-4a38-b17d-a72c264cab38 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:03:25.261136 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 438/876] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Oct 12 00:03:25 2022] GET /identity/v3/users/d6b7beee0c7c40198e7e1403feaa7cf5 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:03:25.266711 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18443721-4040-432c-b49e-9c99e29af0c1 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:03:25.334372 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:03:25.363347 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18443721-4040-432c-b49e-9c99e29af0c1 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:03:25.395861 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8bdd3a-653d-40b6-bce8-cd7d6d185af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:03:25.396455 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8bdd3a-653d-40b6-bce8-cd7d6d185af9 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:25.396730 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8bdd3a-653d-40b6-bce8-cd7d6d185af9 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:25.397000 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8bdd3a-653d-40b6-bce8-cd7d6d185af9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:25.460410 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8bdd3a-653d-40b6-bce8-cd7d6d185af9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:03:25.461834 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8bdd3a-653d-40b6-bce8-cd7d6d185af9 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:03:25.526178 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:03:25.556073 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 439/877] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Oct 12 00:03:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 12 00:03:25.561887 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c712a53a-848d-4a38-b17d-a72c264cab38 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:03:25.658823 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c712a53a-848d-4a38-b17d-a72c264cab38 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:03:25.691100 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f317c39c-b1e4-4362-a7d9-ef2436110843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:03:25.691748 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f317c39c-b1e4-4362-a7d9-ef2436110843 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:25.692027 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f317c39c-b1e4-4362-a7d9-ef2436110843 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:25.692301 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f317c39c-b1e4-4362-a7d9-ef2436110843 None admin] PATH_INFO: `/v3/projects/25a0af5a7de34484a634ceab3ae562e8` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:25.693416 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f317c39c-b1e4-4362-a7d9-ef2436110843 None admin] RBAC: Authorizing `identity:get_project(project_id=25a0af5a7de34484a634ceab3ae562e8)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:03:25.694838 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f317c39c-b1e4-4362-a7d9-ef2436110843 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:03:25.696496 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 439/878] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Oct 12 00:03:25 2022] GET /identity/v3/projects/25a0af5a7de34484a634ceab3ae562e8 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:03:25.703052 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8bdd3a-653d-40b6-bce8-cd7d6d185af9 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:03:25.797927 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8bdd3a-653d-40b6-bce8-cd7d6d185af9 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:03:25.831016 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284253f5-cf74-4c18-8213-361673f19930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:03:25.831627 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284253f5-cf74-4c18-8213-361673f19930 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:25.831904 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284253f5-cf74-4c18-8213-361673f19930 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:25.832170 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284253f5-cf74-4c18-8213-361673f19930 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:25.832880 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284253f5-cf74-4c18-8213-361673f19930 None admin] RBAC: Adding query filter params (role.id=b2e41f0d7a304dc7af8b6a232049b6a2, scope.project.id=25a0af5a7de34484a634ceab3ae562e8, user.id=d6b7beee0c7c40198e7e1403feaa7cf5) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:03:25.833182 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284253f5-cf74-4c18-8213-361673f19930 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:03:25.834620 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284253f5-cf74-4c18-8213-361673f19930 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:03:25.840093 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 440/879] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Oct 12 00:03:25 2022] GET /identity/v3/role_assignments?user.id=d6b7beee0c7c40198e7e1403feaa7cf5&scope.project.id=25a0af5a7de34484a634ceab3ae562e8&role.id=b2e41f0d7a304dc7af8b6a232049b6a2 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:03:27.935837 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a638479d-0047-40ee-bd61-f2daa3f20499 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:27.936369 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a638479d-0047-40ee-bd61-f2daa3f20499 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:27.936369 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a638479d-0047-40ee-bd61-f2daa3f20499 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:27.937706 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 440/880] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:03:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:03:27.942958 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088517a8-eb58-4185-972a-7d290afae31c None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:27.943397 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088517a8-eb58-4185-972a-7d290afae31c None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:27.943527 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088517a8-eb58-4185-972a-7d290afae31c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:27.961691 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-088517a8-eb58-4185-972a-7d290afae31c None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:03:27.962867 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-088517a8-eb58-4185-972a-7d290afae31c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:03:27.989934 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:03:28.028461 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 441/881] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:03:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 00:03:28.295840 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a638479d-0047-40ee-bd61-f2daa3f20499 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:03:28.345318 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:03:28.395455 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a638479d-0047-40ee-bd61-f2daa3f20499 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:03:28.427382 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f6f45c-96fa-420a-9ef0-80409ce7ff61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:03:28.427986 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f6f45c-96fa-420a-9ef0-80409ce7ff61 None admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:28.428265 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f6f45c-96fa-420a-9ef0-80409ce7ff61 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:28.428531 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f6f45c-96fa-420a-9ef0-80409ce7ff61 None admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:28.429195 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f6f45c-96fa-420a-9ef0-80409ce7ff61 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:03:28.430557 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f6f45c-96fa-420a-9ef0-80409ce7ff61 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:03:28.500793 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 441/882] 10.222.0.36 () {60 vars in 1138 bytes} [Wed Oct 12 00:03:28 2022] POST /identity/v3/domains => generated 267 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:03:30.615125 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65c4599-1570-4f4b-a1e7-56c351a63c8d None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:30.615778 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65c4599-1570-4f4b-a1e7-56c351a63c8d None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:30.615778 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65c4599-1570-4f4b-a1e7-56c351a63c8d None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:30.617027 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 442/883] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:03:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:03:30.622118 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fcde35-8910-456c-a17c-1e10a8b5b930 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:30.622538 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fcde35-8910-456c-a17c-1e10a8b5b930 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:30.622655 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fcde35-8910-456c-a17c-1e10a8b5b930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:30.640904 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-04fcde35-8910-456c-a17c-1e10a8b5b930 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:03:30.642069 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-04fcde35-8910-456c-a17c-1e10a8b5b930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:03:30.704962 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 442/884] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:03:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 00:03:30.970956 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65c4599-1570-4f4b-a1e7-56c351a63c8d None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:03:31.068017 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65c4599-1570-4f4b-a1e7-56c351a63c8d None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:03:31.100660 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6daa23-0aae-4a9a-b825-bd5042659af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:03:31.101271 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6daa23-0aae-4a9a-b825-bd5042659af7 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:31.101547 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6daa23-0aae-4a9a-b825-bd5042659af7 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:31.101817 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6daa23-0aae-4a9a-b825-bd5042659af7 None admin] PATH_INFO: `/v3/domains/c640a5aef43c407d84fca9e177ca76de` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:31.111806 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6daa23-0aae-4a9a-b825-bd5042659af7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=c640a5aef43c407d84fca9e177ca76de)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:03:31.113233 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6daa23-0aae-4a9a-b825-bd5042659af7 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:03:31.114783 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 443/885] 10.222.0.36 () {56 vars in 1183 bytes} [Wed Oct 12 00:03:30 2022] GET /identity/v3/domains/c640a5aef43c407d84fca9e177ca76de => generated 267 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:03:31.120470 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fcde35-8910-456c-a17c-1e10a8b5b930 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:03:31.158641 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:03:31.214566 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fcde35-8910-456c-a17c-1e10a8b5b930 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:03:31.246094 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8e60f0-8ea9-4ca1-9665-7ad42075b65b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:03:31.246651 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8e60f0-8ea9-4ca1-9665-7ad42075b65b None admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:31.248008 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8e60f0-8ea9-4ca1-9665-7ad42075b65b None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:31.248276 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8e60f0-8ea9-4ca1-9665-7ad42075b65b None admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:31.248911 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8e60f0-8ea9-4ca1-9665-7ad42075b65b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:03:31.250336 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8e60f0-8ea9-4ca1-9665-7ad42075b65b None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:03:31.366491 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 443/886] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Oct 12 00:03:31 2022] POST /identity/v3/projects => generated 361 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:03:33.510709 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bb566e-69a9-45ad-bb55-39ba4f6b3fb6 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:33.511250 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bb566e-69a9-45ad-bb55-39ba4f6b3fb6 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:33.511250 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bb566e-69a9-45ad-bb55-39ba4f6b3fb6 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:33.512607 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 444/887] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:03:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:03:33.517986 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f571a3-a5e2-4a3e-a169-d6323b90f939 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:33.518402 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f571a3-a5e2-4a3e-a169-d6323b90f939 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:33.518528 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f571a3-a5e2-4a3e-a169-d6323b90f939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:33.536834 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-99f571a3-a5e2-4a3e-a169-d6323b90f939 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:03:33.538030 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-99f571a3-a5e2-4a3e-a169-d6323b90f939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:03:33.602783 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 444/888] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:03:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 00:03:33.866453 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bb566e-69a9-45ad-bb55-39ba4f6b3fb6 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:03:33.894295 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:03:33.964158 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bb566e-69a9-45ad-bb55-39ba4f6b3fb6 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:03:33.997563 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85a67b3-fa9b-4aa9-a25b-e23c2e0a3a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:03:33.998177 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85a67b3-fa9b-4aa9-a25b-e23c2e0a3a7c None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:33.998457 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85a67b3-fa9b-4aa9-a25b-e23c2e0a3a7c None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:33.998730 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85a67b3-fa9b-4aa9-a25b-e23c2e0a3a7c None admin] PATH_INFO: `/v3/domains/c640a5aef43c407d84fca9e177ca76de` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:33.999862 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85a67b3-fa9b-4aa9-a25b-e23c2e0a3a7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=c640a5aef43c407d84fca9e177ca76de)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:03:34.001285 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85a67b3-fa9b-4aa9-a25b-e23c2e0a3a7c None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:03:34.002813 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 445/889] 10.222.0.36 () {56 vars in 1183 bytes} [Wed Oct 12 00:03:33 2022] GET /identity/v3/domains/c640a5aef43c407d84fca9e177ca76de => generated 267 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:03:34.008814 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f571a3-a5e2-4a3e-a169-d6323b90f939 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:03:34.029458 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:03:34.105496 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f571a3-a5e2-4a3e-a169-d6323b90f939 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:03:34.138549 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4985df3-dbb0-447f-862c-ac572f7c1c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:03:34.139084 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4985df3-dbb0-447f-862c-ac572f7c1c6a None admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:34.139382 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4985df3-dbb0-447f-862c-ac572f7c1c6a None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:34.139662 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4985df3-dbb0-447f-862c-ac572f7c1c6a None admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:34.140310 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4985df3-dbb0-447f-862c-ac572f7c1c6a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:03:34.141735 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4985df3-dbb0-447f-862c-ac572f7c1c6a None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:03:34.264792 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 445/890] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Oct 12 00:03:34 2022] POST /identity/v3/users => generated 311 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:03:36.393051 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7458c63-4f78-4ac4-912f-b6652f54f691 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:36.393639 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7458c63-4f78-4ac4-912f-b6652f54f691 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:36.393639 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7458c63-4f78-4ac4-912f-b6652f54f691 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:36.394904 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 446/891] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:03:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:03:36.400284 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1209fcd-68aa-4314-96e0-12a59a270407 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:36.400681 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1209fcd-68aa-4314-96e0-12a59a270407 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:36.400817 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1209fcd-68aa-4314-96e0-12a59a270407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:36.418838 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-b1209fcd-68aa-4314-96e0-12a59a270407 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:03:36.420041 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-b1209fcd-68aa-4314-96e0-12a59a270407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:03:36.484319 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 446/892] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:03:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 00:03:36.749666 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7458c63-4f78-4ac4-912f-b6652f54f691 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:03:36.839439 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:03:36.846339 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7458c63-4f78-4ac4-912f-b6652f54f691 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:03:36.879543 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9288efe1-e93e-42c9-adef-787caf682501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:03:36.880156 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288efe1-e93e-42c9-adef-787caf682501 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:36.880434 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288efe1-e93e-42c9-adef-787caf682501 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:36.880703 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288efe1-e93e-42c9-adef-787caf682501 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:36.890680 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9288efe1-e93e-42c9-adef-787caf682501 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 00:03:36.895491 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-9288efe1-e93e-42c9-adef-787caf682501 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 00:03:36.895491 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:03:36.895491 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:03:36.895491 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:03:36.895491 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:03:36.895491 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:03:36.895491 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:03:36.895491 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:03:36.895491 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:03:36.895491 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:03:36.895491 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:03:36.895491 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:03:36.895491 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 00:03:36.895491 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 00:03:36.895491 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 00:03:36.895491 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 00:03:36.896484 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 00:03:36.896484 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 00:03:36.896484 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 00:03:36.896484 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 00:03:36.896484 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:03:36.896484 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:03:36.896484 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:03:36.896484 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:03:36.896484 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:03:36.896484 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:03:36.896484 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:03:36.896484 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:03:36.896484 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:03:36.896484 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:03:36.896484 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:03:36.896484 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:03:36.897477 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:03:36.897477 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:03:36.897477 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:03:36.897477 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:03:36.897477 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 00:03:36.897477 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 00:03:36.897477 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 00:03:36.897477 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 00:03:36.897477 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 00:03:36.897477 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 00:03:36.897477 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 00:03:36.897477 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:03:36.897477 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 447/893] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Oct 12 00:03:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:03:36.902541 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1209fcd-68aa-4314-96e0-12a59a270407 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:03:36.910657 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:03:36.999024 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1209fcd-68aa-4314-96e0-12a59a270407 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:03:37.031736 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f187ba88-4699-49ee-837f-6e0e60fd2d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:03:37.032275 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f187ba88-4699-49ee-837f-6e0e60fd2d23 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:37.032552 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f187ba88-4699-49ee-837f-6e0e60fd2d23 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:37.032822 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f187ba88-4699-49ee-837f-6e0e60fd2d23 None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:37.033474 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f187ba88-4699-49ee-837f-6e0e60fd2d23 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:03:37.033773 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f187ba88-4699-49ee-837f-6e0e60fd2d23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:03:37.035138 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f187ba88-4699-49ee-837f-6e0e60fd2d23 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:03:37.042893 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 447/894] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Oct 12 00:03:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:03:37.049095 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9288efe1-e93e-42c9-adef-787caf682501 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:03:37.144962 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9288efe1-e93e-42c9-adef-787caf682501 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:03:37.177898 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f813cb-789c-494f-a959-a604cbd0a2a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:03:37.178512 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f813cb-789c-494f-a959-a604cbd0a2a0 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:37.178787 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f813cb-789c-494f-a959-a604cbd0a2a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:37.179057 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f813cb-789c-494f-a959-a604cbd0a2a0 None admin] PATH_INFO: `/v3/users/098159f87bb1427d872e6e7700f2f61c` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:37.191569 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f813cb-789c-494f-a959-a604cbd0a2a0 None admin] RBAC: Authorizing `identity:get_user(user_id=098159f87bb1427d872e6e7700f2f61c)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:03:37.192993 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f813cb-789c-494f-a959-a604cbd0a2a0 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:03:37.194543 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 448/895] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Oct 12 00:03:37 2022] GET /identity/v3/users/098159f87bb1427d872e6e7700f2f61c => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:03:37.200273 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f187ba88-4699-49ee-837f-6e0e60fd2d23 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:03:37.264323 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:03:37.296171 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f187ba88-4699-49ee-837f-6e0e60fd2d23 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:03:37.328538 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc7b5b0-c878-482c-bd69-c4790fdf6722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:03:37.329152 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc7b5b0-c878-482c-bd69-c4790fdf6722 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:37.329428 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc7b5b0-c878-482c-bd69-c4790fdf6722 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:37.329697 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc7b5b0-c878-482c-bd69-c4790fdf6722 None admin] PATH_INFO: `/v3/projects/ea3464a3bbde4af783e763a87eeda93b` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:37.330791 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc7b5b0-c878-482c-bd69-c4790fdf6722 None admin] RBAC: Authorizing `identity:get_project(project_id=ea3464a3bbde4af783e763a87eeda93b)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:03:37.332237 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc7b5b0-c878-482c-bd69-c4790fdf6722 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:03:37.333927 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 448/896] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Oct 12 00:03:37 2022] GET /identity/v3/projects/ea3464a3bbde4af783e763a87eeda93b => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:03:37.339630 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f813cb-789c-494f-a959-a604cbd0a2a0 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:03:37.354315 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:03:37.436065 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f813cb-789c-494f-a959-a604cbd0a2a0 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:03:37.469433 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033c296f-3937-4ef1-a55c-8dc9bb193739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:03:37.470076 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033c296f-3937-4ef1-a55c-8dc9bb193739 None admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:37.470351 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033c296f-3937-4ef1-a55c-8dc9bb193739 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:37.470621 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033c296f-3937-4ef1-a55c-8dc9bb193739 None admin] PATH_INFO: `/v3/projects/ea3464a3bbde4af783e763a87eeda93b/users/098159f87bb1427d872e6e7700f2f61c/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:37.472501 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033c296f-3937-4ef1-a55c-8dc9bb193739 None admin] RBAC: Authorizing `identity:create_grant(project_id=ea3464a3bbde4af783e763a87eeda93b, user_id=098159f87bb1427d872e6e7700f2f61c, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:03:37.473912 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033c296f-3937-4ef1-a55c-8dc9bb193739 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:03:37.538907 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 449/897] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Oct 12 00:03:37 2022] PUT /identity/v3/projects/ea3464a3bbde4af783e763a87eeda93b/users/098159f87bb1427d872e6e7700f2f61c/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:03:39.617549 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9b12fe-13d5-4519-8039-a4f2e09dfb26 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:39.618068 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9b12fe-13d5-4519-8039-a4f2e09dfb26 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:39.618167 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9b12fe-13d5-4519-8039-a4f2e09dfb26 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:39.619455 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 449/898] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:03:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:03:39.625094 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570d9c08-c662-4814-8a15-00d5ddfbe63b None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:39.625459 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570d9c08-c662-4814-8a15-00d5ddfbe63b None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:39.625633 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570d9c08-c662-4814-8a15-00d5ddfbe63b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:39.643467 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-570d9c08-c662-4814-8a15-00d5ddfbe63b None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:03:39.644634 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-570d9c08-c662-4814-8a15-00d5ddfbe63b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:03:39.707307 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 450/899] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:03:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 00:03:39.972037 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9b12fe-13d5-4519-8039-a4f2e09dfb26 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:03:40.069716 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9b12fe-13d5-4519-8039-a4f2e09dfb26 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:03:40.101589 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7564ba9c-5324-42eb-b444-90ef0095ede2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:03:40.102197 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7564ba9c-5324-42eb-b444-90ef0095ede2 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:40.102469 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7564ba9c-5324-42eb-b444-90ef0095ede2 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:40.102733 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7564ba9c-5324-42eb-b444-90ef0095ede2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:40.105217 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:03:40.113391 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7564ba9c-5324-42eb-b444-90ef0095ede2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 00:03:40.118145 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-7564ba9c-5324-42eb-b444-90ef0095ede2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 00:03:40.118145 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:03:40.118145 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:03:40.118145 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:03:40.118145 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:03:40.118145 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:03:40.118145 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:03:40.118145 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:03:40.118145 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:03:40.118145 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:03:40.118145 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:03:40.118145 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:03:40.118145 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 00:03:40.118145 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 00:03:40.118145 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 00:03:40.118145 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 00:03:40.119258 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 00:03:40.119258 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 00:03:40.119258 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 00:03:40.119258 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 00:03:40.119258 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:03:40.119258 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:03:40.119258 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:03:40.119258 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:03:40.119258 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:03:40.119258 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:03:40.119258 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:03:40.119258 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:03:40.119258 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:03:40.119258 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:03:40.119258 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:03:40.119258 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:03:40.120458 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:03:40.120458 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:03:40.120458 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:03:40.120458 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:03:40.120458 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 00:03:40.120458 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 00:03:40.120458 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 00:03:40.120458 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 00:03:40.120458 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 00:03:40.120458 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 00:03:40.120458 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 00:03:40.120458 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:03:40.120458 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 450/900] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Oct 12 00:03:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:03:40.125089 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570d9c08-c662-4814-8a15-00d5ddfbe63b None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:03:40.127230 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:03:40.219732 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570d9c08-c662-4814-8a15-00d5ddfbe63b None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:03:40.251872 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248b90d3-2bd5-48fe-8223-785f0f13b9ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:03:40.252407 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248b90d3-2bd5-48fe-8223-785f0f13b9ee None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:40.252681 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248b90d3-2bd5-48fe-8223-785f0f13b9ee None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:40.252948 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248b90d3-2bd5-48fe-8223-785f0f13b9ee None admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:40.253595 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248b90d3-2bd5-48fe-8223-785f0f13b9ee None admin] RBAC: Adding query filter params (name=admin) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:03:40.253890 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248b90d3-2bd5-48fe-8223-785f0f13b9ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:03:40.255241 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248b90d3-2bd5-48fe-8223-785f0f13b9ee None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:03:40.263152 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 451/901] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Oct 12 00:03:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:03:40.274314 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7564ba9c-5324-42eb-b444-90ef0095ede2 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:03:40.366104 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7564ba9c-5324-42eb-b444-90ef0095ede2 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:03:40.398971 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e747c8-8b62-4d72-b2b0-0095cf28b38f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:03:40.399613 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e747c8-8b62-4d72-b2b0-0095cf28b38f None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:40.399893 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e747c8-8b62-4d72-b2b0-0095cf28b38f None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:40.400161 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e747c8-8b62-4d72-b2b0-0095cf28b38f None admin] PATH_INFO: `/v3/users/098159f87bb1427d872e6e7700f2f61c` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:40.401261 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e747c8-8b62-4d72-b2b0-0095cf28b38f None admin] RBAC: Authorizing `identity:get_user(user_id=098159f87bb1427d872e6e7700f2f61c)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:03:40.403271 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e747c8-8b62-4d72-b2b0-0095cf28b38f None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:03:40.404853 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 451/902] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Oct 12 00:03:40 2022] GET /identity/v3/users/098159f87bb1427d872e6e7700f2f61c => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:03:40.410470 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248b90d3-2bd5-48fe-8223-785f0f13b9ee None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:03:40.470874 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:03:40.507082 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248b90d3-2bd5-48fe-8223-785f0f13b9ee None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:03:40.539597 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b25e37c-6072-4f3b-95e6-28d66efdb441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:03:40.540649 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b25e37c-6072-4f3b-95e6-28d66efdb441 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:40.540649 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b25e37c-6072-4f3b-95e6-28d66efdb441 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:40.540818 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b25e37c-6072-4f3b-95e6-28d66efdb441 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:40.605452 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b25e37c-6072-4f3b-95e6-28d66efdb441 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:03:40.606949 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b25e37c-6072-4f3b-95e6-28d66efdb441 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:03:40.663720 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:03:40.701001 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 452/903] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Oct 12 00:03:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 12 00:03:40.706901 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e747c8-8b62-4d72-b2b0-0095cf28b38f None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:03:40.766560 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:03:40.802588 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e747c8-8b62-4d72-b2b0-0095cf28b38f None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:03:40.834670 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be7ee0d-1fd5-4fe3-ab87-92a0148ae98f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:03:40.835280 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be7ee0d-1fd5-4fe3-ab87-92a0148ae98f None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:40.835580 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be7ee0d-1fd5-4fe3-ab87-92a0148ae98f None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:40.835848 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be7ee0d-1fd5-4fe3-ab87-92a0148ae98f None admin] PATH_INFO: `/v3/projects/ea3464a3bbde4af783e763a87eeda93b` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:40.836952 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be7ee0d-1fd5-4fe3-ab87-92a0148ae98f None admin] RBAC: Authorizing `identity:get_project(project_id=ea3464a3bbde4af783e763a87eeda93b)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:03:40.838361 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be7ee0d-1fd5-4fe3-ab87-92a0148ae98f None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:03:40.839979 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 452/904] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Oct 12 00:03:40 2022] GET /identity/v3/projects/ea3464a3bbde4af783e763a87eeda93b => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:03:40.845954 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b25e37c-6072-4f3b-95e6-28d66efdb441 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:03:40.939974 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b25e37c-6072-4f3b-95e6-28d66efdb441 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:03:40.972109 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752409f3-1fcc-4682-951e-560d446cff43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:03:40.972697 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752409f3-1fcc-4682-951e-560d446cff43 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:03:40.972970 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752409f3-1fcc-4682-951e-560d446cff43 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:03:40.973236 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752409f3-1fcc-4682-951e-560d446cff43 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:03:40.973941 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752409f3-1fcc-4682-951e-560d446cff43 None admin] RBAC: Adding query filter params (role.id=b2e41f0d7a304dc7af8b6a232049b6a2, scope.project.id=ea3464a3bbde4af783e763a87eeda93b, user.id=098159f87bb1427d872e6e7700f2f61c) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:03:40.974243 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752409f3-1fcc-4682-951e-560d446cff43 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:03:40.975673 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752409f3-1fcc-4682-951e-560d446cff43 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:03:40.981165 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 453/905] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Oct 12 00:03:40 2022] GET /identity/v3/role_assignments?user.id=098159f87bb1427d872e6e7700f2f61c&scope.project.id=ea3464a3bbde4af783e763a87eeda93b&role.id=b2e41f0d7a304dc7af8b6a232049b6a2 => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:03:50.830704 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=34 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:03:50.978697 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=86 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:04:52.068894 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c69c8b3-9f7b-4ba4-a69f-a38c4a47ca45 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:04:52.069486 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c69c8b3-9f7b-4ba4-a69f-a38c4a47ca45 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:04:52.069486 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c69c8b3-9f7b-4ba4-a69f-a38c4a47ca45 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:04:52.071405 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 453/906] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:04:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:04:52.076566 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db8da18-7b7c-46ee-b61f-6a809bb892bc None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:04:52.076991 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db8da18-7b7c-46ee-b61f-6a809bb892bc None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:04:52.077091 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db8da18-7b7c-46ee-b61f-6a809bb892bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:04:52.094763 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-0db8da18-7b7c-46ee-b61f-6a809bb892bc None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:04:52.095971 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-0db8da18-7b7c-46ee-b61f-6a809bb892bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:04:52.161267 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 454/907] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:04:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 00:04:52.432276 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c69c8b3-9f7b-4ba4-a69f-a38c4a47ca45 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:04:52.531546 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c69c8b3-9f7b-4ba4-a69f-a38c4a47ca45 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:04:52.567761 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab7abf1-d3d6-4216-b541-3b3003de4319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:04:52.568502 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab7abf1-d3d6-4216-b541-3b3003de4319 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:04:52.568784 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab7abf1-d3d6-4216-b541-3b3003de4319 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:04:52.569052 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab7abf1-d3d6-4216-b541-3b3003de4319 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:04:52.579084 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab7abf1-d3d6-4216-b541-3b3003de4319 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:04:52.580592 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab7abf1-d3d6-4216-b541-3b3003de4319 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:04:52.595030 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-3ab7abf1-d3d6-4216-b541-3b3003de4319 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:04:52.595030 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:04:52.595030 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:04:52.595030 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:04:52.595030 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:04:52.595030 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:04:52.595030 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:04:52.595030 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:04:52.595030 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:04:52.595030 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:04:52.595030 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:04:52.595030 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:04:52.595030 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 00:04:52.595030 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 00:04:52.595030 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 00:04:52.595030 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 00:04:52.596289 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:04:52.596289 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:04:52.596289 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:04:52.596289 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:04:52.596289 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:04:52.596289 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:04:52.596289 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:04:52.596289 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:04:52.596289 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:04:52.596289 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:04:52.596289 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:04:52.596289 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:04:52.596289 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:04:52.596289 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:04:52.596289 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:04:52.596289 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:04:52.596289 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 00:04:52.597325 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 00:04:52.597325 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:04:52.597325 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:04:52.597325 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 454/908] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:04:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:04:52.602875 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db8da18-7b7c-46ee-b61f-6a809bb892bc None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:04:52.700492 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db8da18-7b7c-46ee-b61f-6a809bb892bc None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:04:52.728921 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:04:52.734614 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba83bca0-a191-4218-af7e-d5383d57b9c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:04:52.735216 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba83bca0-a191-4218-af7e-d5383d57b9c0 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:04:52.735527 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba83bca0-a191-4218-af7e-d5383d57b9c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:04:52.735793 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba83bca0-a191-4218-af7e-d5383d57b9c0 None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:04:52.736417 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba83bca0-a191-4218-af7e-d5383d57b9c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:04:52.736712 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba83bca0-a191-4218-af7e-d5383d57b9c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:04:52.738060 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba83bca0-a191-4218-af7e-d5383d57b9c0 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:04:52.748837 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 455/909] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Oct 12 00:04:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:04:52.754517 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab7abf1-d3d6-4216-b541-3b3003de4319 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:04:52.806377 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:04:52.850717 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab7abf1-d3d6-4216-b541-3b3003de4319 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:04:52.883115 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a76349-ee3b-412a-872e-ec84dad97c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:04:52.883736 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a76349-ee3b-412a-872e-ec84dad97c95 None admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:04:52.884012 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a76349-ee3b-412a-872e-ec84dad97c95 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:04:52.884275 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a76349-ee3b-412a-872e-ec84dad97c95 None admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:04:52.884910 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a76349-ee3b-412a-872e-ec84dad97c95 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:04:52.886517 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a76349-ee3b-412a-872e-ec84dad97c95 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:04:52.996543 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 455/910] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Oct 12 00:04:52 2022] POST /identity/v3/users => generated 251 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 00:04:55.131610 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7c632f-4136-4a8a-ae7b-996699e63258 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:04:55.132779 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7c632f-4136-4a8a-ae7b-996699e63258 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:04:55.132779 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7c632f-4136-4a8a-ae7b-996699e63258 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:04:55.134100 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 456/911] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:04:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:04:55.139150 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6adc48-bb80-45ce-85fb-b029158280fc None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:04:55.139575 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6adc48-bb80-45ce-85fb-b029158280fc None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:04:55.139742 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6adc48-bb80-45ce-85fb-b029158280fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:04:55.157843 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-ab6adc48-bb80-45ce-85fb-b029158280fc None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:04:55.159021 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-ab6adc48-bb80-45ce-85fb-b029158280fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:04:55.224006 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 456/912] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:04:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 00:04:55.488579 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7c632f-4136-4a8a-ae7b-996699e63258 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:04:55.591612 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7c632f-4136-4a8a-ae7b-996699e63258 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:04:55.625047 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93684f8b-d9b2-4c62-a967-b4d75d04fbfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:04:55.625661 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93684f8b-d9b2-4c62-a967-b4d75d04fbfe None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:04:55.625934 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93684f8b-d9b2-4c62-a967-b4d75d04fbfe None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:04:55.626200 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93684f8b-d9b2-4c62-a967-b4d75d04fbfe None admin] PATH_INFO: `/v3/roles/service` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:04:55.636564 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93684f8b-d9b2-4c62-a967-b4d75d04fbfe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 00:04:55.641293 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-93684f8b-d9b2-4c62-a967-b4d75d04fbfe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 12 00:04:55.641293 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:04:55.641293 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:04:55.641293 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:04:55.641293 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:04:55.641293 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:04:55.641293 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:04:55.641293 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:04:55.641293 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:04:55.641293 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:04:55.641293 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:04:55.641293 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:04:55.641293 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 00:04:55.641293 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 00:04:55.641293 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 00:04:55.641293 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 00:04:55.643438 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 00:04:55.643438 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 00:04:55.643438 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 00:04:55.643438 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 00:04:55.643438 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:04:55.643438 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:04:55.643438 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:04:55.643438 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:04:55.643438 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:04:55.643438 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:04:55.643438 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:04:55.643438 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:04:55.643438 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:04:55.643438 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:04:55.643438 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:04:55.643438 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:04:55.644366 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:04:55.644366 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:04:55.644366 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:04:55.644366 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:04:55.644366 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 00:04:55.644366 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 00:04:55.644366 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 00:04:55.644366 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 00:04:55.644366 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 00:04:55.644366 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 00:04:55.644366 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 12 00:04:55.644366 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:04:55.644366 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 457/913] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Oct 12 00:04:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:04:55.648429 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6adc48-bb80-45ce-85fb-b029158280fc None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:04:55.687490 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:04:55.745142 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6adc48-bb80-45ce-85fb-b029158280fc None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:04:55.778384 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393637b6-7295-4e22-b5a0-3f846fc865f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:04:55.778983 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393637b6-7295-4e22-b5a0-3f846fc865f0 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:04:55.779262 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393637b6-7295-4e22-b5a0-3f846fc865f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:04:55.779561 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393637b6-7295-4e22-b5a0-3f846fc865f0 None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:04:55.780245 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393637b6-7295-4e22-b5a0-3f846fc865f0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:04:55.780545 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393637b6-7295-4e22-b5a0-3f846fc865f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:04:55.781938 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393637b6-7295-4e22-b5a0-3f846fc865f0 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:04:55.790173 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 457/914] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Oct 12 00:04:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 00:04:55.796087 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93684f8b-d9b2-4c62-a967-b4d75d04fbfe None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:04:55.845662 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:04:55.893147 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93684f8b-d9b2-4c62-a967-b4d75d04fbfe None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:04:55.925608 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95fba3d-8fac-4950-9cd4-4ecb64ddb49f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:04:55.926226 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95fba3d-8fac-4950-9cd4-4ecb64ddb49f None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:04:55.926505 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95fba3d-8fac-4950-9cd4-4ecb64ddb49f None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:04:55.926777 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95fba3d-8fac-4950-9cd4-4ecb64ddb49f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:04:55.938162 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95fba3d-8fac-4950-9cd4-4ecb64ddb49f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:04:55.939668 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95fba3d-8fac-4950-9cd4-4ecb64ddb49f None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:04:55.954499 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-c95fba3d-8fac-4950-9cd4-4ecb64ddb49f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:04:55.954499 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:04:55.954499 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:04:55.954499 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:04:55.954499 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:04:55.954499 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:04:55.954499 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:04:55.954499 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:04:55.954499 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:04:55.954499 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:04:55.954499 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:04:55.954499 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:04:55.954499 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 00:04:55.954499 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 00:04:55.954499 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 00:04:55.954499 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 00:04:55.955715 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:04:55.955715 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:04:55.955715 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:04:55.955715 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:04:55.955715 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:04:55.955715 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:04:55.955715 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:04:55.955715 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:04:55.955715 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:04:55.955715 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:04:55.955715 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:04:55.955715 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:04:55.955715 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:04:55.955715 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:04:55.955715 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:04:55.955715 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:04:55.955715 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 00:04:55.956661 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 00:04:55.956661 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:04:55.956661 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:04:55.956661 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 458/915] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:04:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:04:55.961875 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393637b6-7295-4e22-b5a0-3f846fc865f0 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:04:56.058007 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:04:56.060803 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393637b6-7295-4e22-b5a0-3f846fc865f0 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:04:56.093462 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26671379-021c-4103-8647-575b3d547f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:04:56.094062 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26671379-021c-4103-8647-575b3d547f2b None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:04:56.094338 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26671379-021c-4103-8647-575b3d547f2b None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:04:56.094605 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26671379-021c-4103-8647-575b3d547f2b None admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:04:56.095238 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26671379-021c-4103-8647-575b3d547f2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:04:56.095564 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26671379-021c-4103-8647-575b3d547f2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:04:56.096916 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26671379-021c-4103-8647-575b3d547f2b None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:04:56.107537 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 458/916] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Oct 12 00:04:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:04:56.113163 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95fba3d-8fac-4950-9cd4-4ecb64ddb49f None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:04:56.213147 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95fba3d-8fac-4950-9cd4-4ecb64ddb49f None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:04:56.214671 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:04:56.245680 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b2693f-3f99-472a-aa96-035386935f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:04:56.246295 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b2693f-3f99-472a-aa96-035386935f38 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:04:56.246565 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b2693f-3f99-472a-aa96-035386935f38 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:04:56.246830 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b2693f-3f99-472a-aa96-035386935f38 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:04:56.253632 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b2693f-3f99-472a-aa96-035386935f38 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:04:56.255344 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b2693f-3f99-472a-aa96-035386935f38 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:04:56.266896 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-79b2693f-3f99-472a-aa96-035386935f38 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 12 00:04:56.266896 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:04:56.266896 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:04:56.266896 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:04:56.266896 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:04:56.266896 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:04:56.266896 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:04:56.266896 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:04:56.266896 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:04:56.266896 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:04:56.266896 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:04:56.266896 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:04:56.266896 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 00:04:56.266896 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 00:04:56.266896 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 00:04:56.266896 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 00:04:56.268048 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:04:56.268048 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:04:56.268048 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 00:04:56.268048 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 00:04:56.268048 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 00:04:56.268048 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 00:04:56.268048 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:04:56.268048 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:04:56.268048 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:04:56.268048 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:04:56.268048 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:04:56.268048 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:04:56.268048 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:04:56.268048 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:04:56.268048 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:04:56.268048 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:04:56.268048 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:04:56.268048 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:04:56.268978 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:04:56.268978 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:04:56.268978 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 00:04:56.268978 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 00:04:56.268978 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 00:04:56.268978 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 00:04:56.268978 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 00:04:56.268978 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 00:04:56.268978 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 12 00:04:56.268978 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:04:56.268978 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 459/917] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:04:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:04:56.274471 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26671379-021c-4103-8647-575b3d547f2b None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:04:56.370789 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26671379-021c-4103-8647-575b3d547f2b None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:04:56.404836 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee04d9ad-0c4f-4676-b0fb-d17eb32725fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:04:56.405433 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee04d9ad-0c4f-4676-b0fb-d17eb32725fa None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:04:56.405708 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee04d9ad-0c4f-4676-b0fb-d17eb32725fa None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:04:56.405975 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee04d9ad-0c4f-4676-b0fb-d17eb32725fa None admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:04:56.406685 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee04d9ad-0c4f-4676-b0fb-d17eb32725fa None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:04:56.406982 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee04d9ad-0c4f-4676-b0fb-d17eb32725fa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:04:56.408436 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee04d9ad-0c4f-4676-b0fb-d17eb32725fa None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:04:56.424919 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 459/918] 10.222.0.36 () {56 vars in 1143 bytes} [Wed Oct 12 00:04:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:04:56.431435 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b2693f-3f99-472a-aa96-035386935f38 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:04:56.529151 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b2693f-3f99-472a-aa96-035386935f38 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:04:56.563322 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f01f7b7-cc0d-4a7c-bd3b-933fa80b71f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:04:56.563967 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f01f7b7-cc0d-4a7c-bd3b-933fa80b71f9 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:04:56.564253 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f01f7b7-cc0d-4a7c-bd3b-933fa80b71f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:04:56.564525 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f01f7b7-cc0d-4a7c-bd3b-933fa80b71f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:04:56.575456 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f01f7b7-cc0d-4a7c-bd3b-933fa80b71f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:04:56.576946 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f01f7b7-cc0d-4a7c-bd3b-933fa80b71f9 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:04:56.590812 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-4f01f7b7-cc0d-4a7c-bd3b-933fa80b71f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:04:56.590812 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:04:56.590812 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:04:56.590812 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:04:56.590812 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:04:56.590812 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:04:56.590812 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:04:56.590812 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:04:56.590812 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:04:56.590812 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:04:56.590812 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:04:56.590812 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:04:56.590812 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 00:04:56.590812 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 00:04:56.590812 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 00:04:56.590812 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 00:04:56.591955 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:04:56.591955 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:04:56.591955 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:04:56.591955 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:04:56.591955 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:04:56.591955 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:04:56.591955 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:04:56.591955 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:04:56.591955 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:04:56.591955 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:04:56.591955 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:04:56.591955 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:04:56.591955 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:04:56.591955 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:04:56.591955 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:04:56.591955 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:04:56.591955 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 00:04:56.593121 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 00:04:56.593121 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:04:56.593121 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:04:56.593121 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 460/919] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:04:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:04:56.597741 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee04d9ad-0c4f-4676-b0fb-d17eb32725fa None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:04:56.616088 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:04:56.694079 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee04d9ad-0c4f-4676-b0fb-d17eb32725fa None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:04:56.729881 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c5da4d-7443-4f1d-a503-f1f538b31581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:04:56.730489 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c5da4d-7443-4f1d-a503-f1f538b31581 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:04:56.730768 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c5da4d-7443-4f1d-a503-f1f538b31581 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:04:56.731039 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c5da4d-7443-4f1d-a503-f1f538b31581 None admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:04:56.731735 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c5da4d-7443-4f1d-a503-f1f538b31581 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:04:56.732036 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c5da4d-7443-4f1d-a503-f1f538b31581 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:04:56.733407 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c5da4d-7443-4f1d-a503-f1f538b31581 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:04:56.745018 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 460/920] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Oct 12 00:04:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:04:56.750916 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f01f7b7-cc0d-4a7c-bd3b-933fa80b71f9 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:04:56.767977 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:04:56.850349 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f01f7b7-cc0d-4a7c-bd3b-933fa80b71f9 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:04:56.883775 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf397001-fc61-4ce5-ad14-09e4849aa80a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:04:56.884392 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf397001-fc61-4ce5-ad14-09e4849aa80a None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:04:56.884668 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf397001-fc61-4ce5-ad14-09e4849aa80a None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:04:56.884936 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf397001-fc61-4ce5-ad14-09e4849aa80a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:04:56.891440 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf397001-fc61-4ce5-ad14-09e4849aa80a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:04:56.892872 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf397001-fc61-4ce5-ad14-09e4849aa80a None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:04:56.903839 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-cf397001-fc61-4ce5-ad14-09e4849aa80a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 00:04:56.903839 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:04:56.903839 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:04:56.903839 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:04:56.903839 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:04:56.903839 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:04:56.903839 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:04:56.903839 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:04:56.903839 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:04:56.903839 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:04:56.903839 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:04:56.903839 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:04:56.903839 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 00:04:56.903839 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 00:04:56.903839 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 00:04:56.904954 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 00:04:56.904954 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:04:56.904954 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:04:56.904954 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:04:56.904954 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:04:56.904954 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:04:56.904954 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:04:56.904954 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:04:56.904954 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:04:56.904954 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:04:56.904954 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:04:56.904954 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:04:56.904954 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:04:56.904954 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:04:56.904954 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:04:56.904954 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:04:56.904954 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:04:56.911783 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 00:04:56.911783 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 00:04:56.911783 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 00:04:56.911783 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 00:04:56.911783 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 00:04:56.911783 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 00:04:56.911783 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 00:04:56.911783 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:04:56.911783 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 461/921] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Oct 12 00:04:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:04:56.912412 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c5da4d-7443-4f1d-a503-f1f538b31581 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:04:56.988322 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:04:57.010245 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c5da4d-7443-4f1d-a503-f1f538b31581 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:04:57.043263 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e47006-45a3-40cc-835b-63e0343ce208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:04:57.043892 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e47006-45a3-40cc-835b-63e0343ce208 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:04:57.044173 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e47006-45a3-40cc-835b-63e0343ce208 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:04:57.044442 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e47006-45a3-40cc-835b-63e0343ce208 None admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:04:57.045183 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e47006-45a3-40cc-835b-63e0343ce208 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:04:57.045482 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e47006-45a3-40cc-835b-63e0343ce208 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:04:57.046908 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e47006-45a3-40cc-835b-63e0343ce208 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:04:57.059150 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 461/922] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Oct 12 00:04:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:04:57.065281 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf397001-fc61-4ce5-ad14-09e4849aa80a None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:04:57.138803 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:04:57.162130 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf397001-fc61-4ce5-ad14-09e4849aa80a None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:04:57.194978 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7d097a-4e50-477a-890c-2216c71510e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:04:57.195750 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7d097a-4e50-477a-890c-2216c71510e4 None admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:04:57.196026 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7d097a-4e50-477a-890c-2216c71510e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:04:57.196292 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7d097a-4e50-477a-890c-2216c71510e4 None admin] PATH_INFO: `/v3/projects/0697927c6aac43d4b14a750fbbb1145e/users/1e528da752ed4c78bc747215909f6569/roles/a53c0c4b8b8645b6a1cfe96266daec8e` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:04:57.211814 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7d097a-4e50-477a-890c-2216c71510e4 None admin] RBAC: Authorizing `identity:create_grant(project_id=0697927c6aac43d4b14a750fbbb1145e, user_id=1e528da752ed4c78bc747215909f6569, role_id=a53c0c4b8b8645b6a1cfe96266daec8e)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:04:57.213253 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7d097a-4e50-477a-890c-2216c71510e4 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:04:57.295146 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 462/923] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Oct 12 00:04:57 2022] PUT /identity/v3/projects/0697927c6aac43d4b14a750fbbb1145e/users/1e528da752ed4c78bc747215909f6569/roles/a53c0c4b8b8645b6a1cfe96266daec8e => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:04:59.431247 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b7ce80-7de7-4bc0-b36d-ea0e8d9b2a1f None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:04:59.431848 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b7ce80-7de7-4bc0-b36d-ea0e8d9b2a1f None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:04:59.431848 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b7ce80-7de7-4bc0-b36d-ea0e8d9b2a1f None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:04:59.433189 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 462/924] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:04:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 00:04:59.438319 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9635aed-6459-4812-9b1b-3e2e569eaa46 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:04:59.438728 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9635aed-6459-4812-9b1b-3e2e569eaa46 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:04:59.438870 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9635aed-6459-4812-9b1b-3e2e569eaa46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:04:59.457348 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-b9635aed-6459-4812-9b1b-3e2e569eaa46 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:04:59.458539 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-b9635aed-6459-4812-9b1b-3e2e569eaa46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:04:59.524321 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 463/925] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:04:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 00:04:59.796290 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b7ce80-7de7-4bc0-b36d-ea0e8d9b2a1f None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:04:59.896400 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b7ce80-7de7-4bc0-b36d-ea0e8d9b2a1f None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:04:59.930052 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdc7b0e-dab1-4f4b-a1ad-1a655d9acd7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:04:59.930674 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdc7b0e-dab1-4f4b-a1ad-1a655d9acd7f None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:04:59.930956 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdc7b0e-dab1-4f4b-a1ad-1a655d9acd7f None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:04:59.931329 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdc7b0e-dab1-4f4b-a1ad-1a655d9acd7f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:04:59.937864 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:04:59.941650 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdc7b0e-dab1-4f4b-a1ad-1a655d9acd7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 00:04:59.946500 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-fcdc7b0e-dab1-4f4b-a1ad-1a655d9acd7f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 12 00:04:59.946500 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:04:59.946500 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:04:59.946500 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:04:59.946500 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:04:59.946500 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:04:59.946500 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:04:59.946500 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:04:59.946500 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:04:59.946500 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:04:59.946500 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:04:59.946500 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:04:59.946500 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 00:04:59.946500 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 00:04:59.946500 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 00:04:59.946500 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 00:04:59.947515 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 00:04:59.947515 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 00:04:59.947515 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 00:04:59.947515 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 00:04:59.947515 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:04:59.947515 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:04:59.947515 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:04:59.947515 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:04:59.947515 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:04:59.947515 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:04:59.947515 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:04:59.947515 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:04:59.947515 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:04:59.947515 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:04:59.947515 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:04:59.947515 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:04:59.953719 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:04:59.953719 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:04:59.953719 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:04:59.953719 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:04:59.953719 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 00:04:59.953719 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 00:04:59.953719 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 00:04:59.953719 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 00:04:59.953719 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 00:04:59.953719 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 00:04:59.953719 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 12 00:04:59.953719 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:04:59.953719 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 463/926] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Oct 12 00:04:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:04:59.954573 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9635aed-6459-4812-9b1b-3e2e569eaa46 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:05:00.007504 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:05:00.049806 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9635aed-6459-4812-9b1b-3e2e569eaa46 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:05:00.082700 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ef4e55-e6e7-4e42-884a-e86e469f64ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:05:00.083276 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ef4e55-e6e7-4e42-884a-e86e469f64ee None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:05:00.083586 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ef4e55-e6e7-4e42-884a-e86e469f64ee None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:05:00.083855 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ef4e55-e6e7-4e42-884a-e86e469f64ee None admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:05:00.084542 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ef4e55-e6e7-4e42-884a-e86e469f64ee None admin] RBAC: Adding query filter params (name=service) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:05:00.089848 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ef4e55-e6e7-4e42-884a-e86e469f64ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:05:00.089848 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ef4e55-e6e7-4e42-884a-e86e469f64ee None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:05:00.094429 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 464/927] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Oct 12 00:04:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:05:00.100397 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdc7b0e-dab1-4f4b-a1ad-1a655d9acd7f None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:05:00.199769 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdc7b0e-dab1-4f4b-a1ad-1a655d9acd7f None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:05:00.233600 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a74b9a-23bc-4853-9741-825ef418c37a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:05:00.234217 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a74b9a-23bc-4853-9741-825ef418c37a None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:05:00.234490 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a74b9a-23bc-4853-9741-825ef418c37a None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:05:00.234756 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a74b9a-23bc-4853-9741-825ef418c37a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:05:00.244642 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a74b9a-23bc-4853-9741-825ef418c37a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:05:00.246075 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a74b9a-23bc-4853-9741-825ef418c37a None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:05:00.260643 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-f4a74b9a-23bc-4853-9741-825ef418c37a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:05:00.260643 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:05:00.260643 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:05:00.260643 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:05:00.260643 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:05:00.260643 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:05:00.260643 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:05:00.260643 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:05:00.260643 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:05:00.260643 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:05:00.260643 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:05:00.260643 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:05:00.260643 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 00:05:00.260643 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 00:05:00.260643 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 00:05:00.260643 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 00:05:00.261769 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:05:00.261769 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:05:00.261769 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:05:00.261769 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:05:00.261769 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:05:00.261769 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:05:00.261769 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:05:00.261769 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:05:00.261769 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:05:00.261769 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:05:00.261769 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:05:00.261769 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:05:00.261769 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:05:00.261769 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:05:00.261769 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:05:00.261769 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:05:00.261769 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 00:05:00.262901 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 00:05:00.262901 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:05:00.262901 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:05:00.262901 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 464/928] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:05:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 00:05:00.267647 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ef4e55-e6e7-4e42-884a-e86e469f64ee None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:05:00.365808 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ef4e55-e6e7-4e42-884a-e86e469f64ee None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:05:00.383722 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:05:00.400723 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c600d287-c697-474e-aa03-5f4677ab5bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:05:00.401323 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c600d287-c697-474e-aa03-5f4677ab5bbf None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:05:00.401596 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c600d287-c697-474e-aa03-5f4677ab5bbf None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:05:00.401858 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c600d287-c697-474e-aa03-5f4677ab5bbf None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:05:00.402519 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c600d287-c697-474e-aa03-5f4677ab5bbf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:05:00.402811 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c600d287-c697-474e-aa03-5f4677ab5bbf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:05:00.404195 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c600d287-c697-474e-aa03-5f4677ab5bbf None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:05:00.415409 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 465/929] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Oct 12 00:05:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:05:00.421312 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a74b9a-23bc-4853-9741-825ef418c37a None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:05:00.471826 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:05:00.520665 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a74b9a-23bc-4853-9741-825ef418c37a None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:05:00.555451 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb51de28-d21b-4ed4-b883-3f54c9613a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:05:00.556078 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb51de28-d21b-4ed4-b883-3f54c9613a23 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:05:00.556350 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb51de28-d21b-4ed4-b883-3f54c9613a23 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:05:00.556631 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb51de28-d21b-4ed4-b883-3f54c9613a23 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:05:00.564049 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb51de28-d21b-4ed4-b883-3f54c9613a23 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:05:00.565477 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb51de28-d21b-4ed4-b883-3f54c9613a23 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:05:00.577203 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-bb51de28-d21b-4ed4-b883-3f54c9613a23 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 12 00:05:00.577203 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:05:00.577203 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:05:00.577203 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:05:00.577203 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:05:00.577203 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:05:00.577203 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:05:00.577203 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:05:00.577203 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:05:00.577203 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:05:00.577203 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:05:00.577203 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:05:00.577203 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 00:05:00.577203 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 00:05:00.577203 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 00:05:00.577203 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 00:05:00.578345 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:05:00.578345 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:05:00.578345 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 00:05:00.578345 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 00:05:00.578345 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 00:05:00.578345 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 00:05:00.578345 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:05:00.578345 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:05:00.578345 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:05:00.578345 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:05:00.578345 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:05:00.578345 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:05:00.578345 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:05:00.578345 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:05:00.578345 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:05:00.578345 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:05:00.578345 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:05:00.578345 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:05:00.579620 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:05:00.579620 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:05:00.579620 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 00:05:00.579620 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 00:05:00.579620 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 00:05:00.579620 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 00:05:00.579620 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 00:05:00.579620 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 00:05:00.579620 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 12 00:05:00.579620 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:05:00.579620 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 465/930] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:05:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:05:00.584390 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c600d287-c697-474e-aa03-5f4677ab5bbf None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:05:00.682079 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c600d287-c697-474e-aa03-5f4677ab5bbf None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:05:00.714250 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72372917-8c79-4fdd-ada3-22a1f8704f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:05:00.714851 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72372917-8c79-4fdd-ada3-22a1f8704f91 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:05:00.715126 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72372917-8c79-4fdd-ada3-22a1f8704f91 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:05:00.715412 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72372917-8c79-4fdd-ada3-22a1f8704f91 None admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:05:00.716256 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72372917-8c79-4fdd-ada3-22a1f8704f91 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:05:00.716556 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72372917-8c79-4fdd-ada3-22a1f8704f91 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:05:00.717977 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72372917-8c79-4fdd-ada3-22a1f8704f91 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:05:00.735034 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 466/931] 10.222.0.36 () {56 vars in 1143 bytes} [Wed Oct 12 00:05:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:05:00.740923 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb51de28-d21b-4ed4-b883-3f54c9613a23 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:05:00.840935 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb51de28-d21b-4ed4-b883-3f54c9613a23 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:05:00.852624 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:05:00.875790 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaa2125-4130-4073-925e-36222b601e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:05:00.876440 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaa2125-4130-4073-925e-36222b601e8e None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:05:00.876717 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaa2125-4130-4073-925e-36222b601e8e None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:05:00.876988 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaa2125-4130-4073-925e-36222b601e8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:05:00.941997 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaa2125-4130-4073-925e-36222b601e8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:05:00.943394 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaa2125-4130-4073-925e-36222b601e8e None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:05:01.037237 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 466/932] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Oct 12 00:05:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 12 00:05:01.043781 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72372917-8c79-4fdd-ada3-22a1f8704f91 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:05:01.077414 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:05:01.140518 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72372917-8c79-4fdd-ada3-22a1f8704f91 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:05:01.175415 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639b1e25-26df-473d-8b8f-0be2473582a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:05:01.176036 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b1e25-26df-473d-8b8f-0be2473582a2 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:05:01.176309 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b1e25-26df-473d-8b8f-0be2473582a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:05:01.176577 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b1e25-26df-473d-8b8f-0be2473582a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:05:01.186370 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639b1e25-26df-473d-8b8f-0be2473582a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:05:01.187971 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639b1e25-26df-473d-8b8f-0be2473582a2 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:05:01.202061 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-639b1e25-26df-473d-8b8f-0be2473582a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:05:01.202061 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:05:01.202061 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:05:01.202061 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:05:01.202061 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:05:01.202061 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:05:01.202061 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:05:01.202061 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:05:01.202061 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:05:01.202061 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:05:01.202061 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:05:01.202061 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:05:01.202061 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 00:05:01.202061 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 00:05:01.202061 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 00:05:01.202061 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 00:05:01.202988 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:05:01.202988 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:05:01.202988 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:05:01.202988 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:05:01.202988 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:05:01.202988 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:05:01.202988 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:05:01.202988 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:05:01.202988 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:05:01.202988 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:05:01.202988 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:05:01.202988 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:05:01.202988 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:05:01.202988 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:05:01.202988 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:05:01.202988 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:05:01.202988 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 00:05:01.204019 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 00:05:01.204019 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:05:01.204019 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:05:01.204019 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 467/933] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:05:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:05:01.208883 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaa2125-4130-4073-925e-36222b601e8e None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:05:01.220731 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:05:01.306652 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaa2125-4130-4073-925e-36222b601e8e None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:05:01.340381 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2952a9a-ff32-449e-8fda-2fec4bca9266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:05:01.340981 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2952a9a-ff32-449e-8fda-2fec4bca9266 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:05:01.341257 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2952a9a-ff32-449e-8fda-2fec4bca9266 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:05:01.341527 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2952a9a-ff32-449e-8fda-2fec4bca9266 None admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:05:01.342163 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2952a9a-ff32-449e-8fda-2fec4bca9266 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:05:01.342454 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2952a9a-ff32-449e-8fda-2fec4bca9266 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:05:01.343922 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2952a9a-ff32-449e-8fda-2fec4bca9266 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:05:01.354710 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 467/934] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Oct 12 00:05:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:05:01.360543 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639b1e25-26df-473d-8b8f-0be2473582a2 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:05:01.442644 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:05:01.457689 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639b1e25-26df-473d-8b8f-0be2473582a2 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:05:01.490153 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20ddde7-1a01-4ef1-b239-f2dcfd4e47e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:05:01.490783 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20ddde7-1a01-4ef1-b239-f2dcfd4e47e6 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:05:01.491064 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20ddde7-1a01-4ef1-b239-f2dcfd4e47e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:05:01.491336 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20ddde7-1a01-4ef1-b239-f2dcfd4e47e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:05:01.497799 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20ddde7-1a01-4ef1-b239-f2dcfd4e47e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:05:01.499292 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20ddde7-1a01-4ef1-b239-f2dcfd4e47e6 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:05:01.511203 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-a20ddde7-1a01-4ef1-b239-f2dcfd4e47e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 00:05:01.511203 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:05:01.511203 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:05:01.511203 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:05:01.511203 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:05:01.511203 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:05:01.511203 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:05:01.511203 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:05:01.511203 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:05:01.511203 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:05:01.511203 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:05:01.511203 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:05:01.511203 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 00:05:01.511203 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 00:05:01.511203 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 00:05:01.512163 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 00:05:01.512163 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:05:01.512163 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:05:01.512163 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:05:01.512163 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:05:01.512163 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:05:01.512163 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:05:01.512163 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:05:01.512163 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:05:01.512163 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:05:01.512163 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:05:01.512163 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:05:01.512163 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:05:01.512163 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:05:01.512163 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:05:01.512163 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:05:01.512163 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:05:01.513139 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 00:05:01.513139 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 00:05:01.513139 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 00:05:01.513139 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 00:05:01.513139 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 00:05:01.513139 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 00:05:01.513139 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 00:05:01.513139 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:05:01.513139 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 468/935] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Oct 12 00:05:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:05:01.518303 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2952a9a-ff32-449e-8fda-2fec4bca9266 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:05:01.587634 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:05:01.616710 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2952a9a-ff32-449e-8fda-2fec4bca9266 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:05:01.650104 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a715bea0-b0e3-401c-9573-a5858df11f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:05:01.650706 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a715bea0-b0e3-401c-9573-a5858df11f1b None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:05:01.651112 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a715bea0-b0e3-401c-9573-a5858df11f1b None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:05:01.651395 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a715bea0-b0e3-401c-9573-a5858df11f1b None admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:05:01.652096 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a715bea0-b0e3-401c-9573-a5858df11f1b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:05:01.652398 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a715bea0-b0e3-401c-9573-a5858df11f1b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:05:01.653829 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a715bea0-b0e3-401c-9573-a5858df11f1b None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:05:01.665290 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 468/936] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Oct 12 00:05:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:05:01.671685 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20ddde7-1a01-4ef1-b239-f2dcfd4e47e6 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:05:01.769625 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20ddde7-1a01-4ef1-b239-f2dcfd4e47e6 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:05:01.801042 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e218ea-d99b-4a84-9e4e-e22b537b1a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:05:01.801638 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e218ea-d99b-4a84-9e4e-e22b537b1a17 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:05:01.801921 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e218ea-d99b-4a84-9e4e-e22b537b1a17 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:05:01.802182 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e218ea-d99b-4a84-9e4e-e22b537b1a17 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:05:01.802892 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e218ea-d99b-4a84-9e4e-e22b537b1a17 None admin] RBAC: Adding query filter params (role.id=a53c0c4b8b8645b6a1cfe96266daec8e, scope.project.id=0697927c6aac43d4b14a750fbbb1145e, user.id=1e528da752ed4c78bc747215909f6569) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:05:01.803193 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e218ea-d99b-4a84-9e4e-e22b537b1a17 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:05:01.805144 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e218ea-d99b-4a84-9e4e-e22b537b1a17 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:05:01.810735 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 469/937] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Oct 12 00:05:01 2022] GET /identity/v3/role_assignments?user.id=1e528da752ed4c78bc747215909f6569&scope.project.id=0697927c6aac43d4b14a750fbbb1145e&role.id=a53c0c4b8b8645b6a1cfe96266daec8e => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:05:03.949122 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c641dfe5-dd0a-47f8-9020-89694c37f41c None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:05:03.949652 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c641dfe5-dd0a-47f8-9020-89694c37f41c None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:05:03.949652 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c641dfe5-dd0a-47f8-9020-89694c37f41c None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:05:03.951029 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 469/938] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:05:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:05:03.956335 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee3db75-9451-494e-8066-4f78cbb70361 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:05:03.956754 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee3db75-9451-494e-8066-4f78cbb70361 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:05:03.956855 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee3db75-9451-494e-8066-4f78cbb70361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:05:03.976002 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-dee3db75-9451-494e-8066-4f78cbb70361 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:05:03.977217 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-dee3db75-9451-494e-8066-4f78cbb70361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:05:03.979983 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:05:04.041809 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 470/939] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:05:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 00:05:04.308191 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c641dfe5-dd0a-47f8-9020-89694c37f41c None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:05:04.409432 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c641dfe5-dd0a-47f8-9020-89694c37f41c None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:05:04.437203 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:05:04.442859 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf450fde-d1a3-41c7-bdf6-2dc8bc07578a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:05:04.443532 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf450fde-d1a3-41c7-bdf6-2dc8bc07578a None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:05:04.443814 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf450fde-d1a3-41c7-bdf6-2dc8bc07578a None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:05:04.444600 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf450fde-d1a3-41c7-bdf6-2dc8bc07578a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:05:04.454680 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf450fde-d1a3-41c7-bdf6-2dc8bc07578a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 00:05:04.460153 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-bf450fde-d1a3-41c7-bdf6-2dc8bc07578a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 00:05:04.460153 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:05:04.460153 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:05:04.460153 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:05:04.460153 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:05:04.460153 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:05:04.460153 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:05:04.460153 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:05:04.460153 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:05:04.460153 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:05:04.460153 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:05:04.460153 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:05:04.460153 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 00:05:04.460153 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 00:05:04.460153 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 00:05:04.460153 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 00:05:04.461338 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 00:05:04.461338 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 00:05:04.461338 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 00:05:04.461338 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 00:05:04.461338 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:05:04.461338 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:05:04.461338 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:05:04.461338 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:05:04.461338 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:05:04.461338 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:05:04.461338 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:05:04.461338 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:05:04.461338 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:05:04.461338 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:05:04.461338 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:05:04.461338 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:05:04.462163 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:05:04.462163 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:05:04.462163 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:05:04.462163 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:05:04.462163 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 00:05:04.462163 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 00:05:04.462163 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 00:05:04.462163 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 00:05:04.462163 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 00:05:04.462163 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 00:05:04.462163 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 00:05:04.462163 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:05:04.462163 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 470/940] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Oct 12 00:05:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:05:04.467299 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee3db75-9451-494e-8066-4f78cbb70361 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:05:04.563130 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee3db75-9451-494e-8066-4f78cbb70361 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:05:04.596026 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1646c2-63b6-4c05-9080-30a25b24d23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:05:04.596621 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1646c2-63b6-4c05-9080-30a25b24d23a None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:05:04.596894 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1646c2-63b6-4c05-9080-30a25b24d23a None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:05:04.597158 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1646c2-63b6-4c05-9080-30a25b24d23a None admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:05:04.597797 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1646c2-63b6-4c05-9080-30a25b24d23a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:05:04.598092 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1646c2-63b6-4c05-9080-30a25b24d23a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:05:04.599469 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1646c2-63b6-4c05-9080-30a25b24d23a None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:05:04.607149 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 471/941] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Oct 12 00:05:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:05:04.613034 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf450fde-d1a3-41c7-bdf6-2dc8bc07578a None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:05:04.710676 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf450fde-d1a3-41c7-bdf6-2dc8bc07578a None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:05:04.743506 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3921f15a-0910-4ea7-9aff-d7fbc019d9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:05:04.744152 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3921f15a-0910-4ea7-9aff-d7fbc019d9d3 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:05:04.744426 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3921f15a-0910-4ea7-9aff-d7fbc019d9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:05:04.744692 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3921f15a-0910-4ea7-9aff-d7fbc019d9d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:05:04.754870 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3921f15a-0910-4ea7-9aff-d7fbc019d9d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:05:04.756374 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3921f15a-0910-4ea7-9aff-d7fbc019d9d3 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:05:04.771695 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-3921f15a-0910-4ea7-9aff-d7fbc019d9d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:05:04.771695 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:05:04.771695 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:05:04.771695 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:05:04.771695 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:05:04.771695 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:05:04.771695 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:05:04.771695 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:05:04.771695 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:05:04.771695 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:05:04.771695 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:05:04.771695 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:05:04.771695 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 00:05:04.771695 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 00:05:04.771695 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 00:05:04.771695 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 00:05:04.773332 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:05:04.773332 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:05:04.773332 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:05:04.773332 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:05:04.773332 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:05:04.773332 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:05:04.773332 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:05:04.773332 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:05:04.773332 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:05:04.773332 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:05:04.773332 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:05:04.773332 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:05:04.773332 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:05:04.773332 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:05:04.773332 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:05:04.773332 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:05:04.773332 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 00:05:04.774375 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 00:05:04.774375 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:05:04.774375 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:05:04.774375 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 471/942] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:05:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:05:04.778436 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1646c2-63b6-4c05-9080-30a25b24d23a None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:05:04.781289 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:05:04.875500 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1646c2-63b6-4c05-9080-30a25b24d23a None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:05:04.907867 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac14d0b-04f8-4cbd-bb66-459bfd29d5e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:05:04.908455 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac14d0b-04f8-4cbd-bb66-459bfd29d5e1 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:05:04.908727 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac14d0b-04f8-4cbd-bb66-459bfd29d5e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:05:04.908992 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac14d0b-04f8-4cbd-bb66-459bfd29d5e1 None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:05:04.909614 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac14d0b-04f8-4cbd-bb66-459bfd29d5e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:05:04.909907 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac14d0b-04f8-4cbd-bb66-459bfd29d5e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:05:04.911261 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac14d0b-04f8-4cbd-bb66-459bfd29d5e1 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:05:04.922591 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 472/943] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Oct 12 00:05:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:05:04.928211 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3921f15a-0910-4ea7-9aff-d7fbc019d9d3 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:05:04.971620 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:05:05.026611 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3921f15a-0910-4ea7-9aff-d7fbc019d9d3 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:05:05.060288 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80a488a-e7f9-4386-8efc-1f4e1d57d36d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:05:05.060908 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80a488a-e7f9-4386-8efc-1f4e1d57d36d None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:05:05.061180 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80a488a-e7f9-4386-8efc-1f4e1d57d36d None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:05:05.061443 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80a488a-e7f9-4386-8efc-1f4e1d57d36d None admin] PATH_INFO: `/v3/users/placement` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:05:05.068214 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80a488a-e7f9-4386-8efc-1f4e1d57d36d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:05:05.069582 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80a488a-e7f9-4386-8efc-1f4e1d57d36d None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:05:05.080818 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-d80a488a-e7f9-4386-8efc-1f4e1d57d36d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 12 00:05:05.080818 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:05:05.080818 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:05:05.080818 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:05:05.080818 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:05:05.080818 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:05:05.080818 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:05:05.080818 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:05:05.080818 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:05:05.080818 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:05:05.080818 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:05:05.080818 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:05:05.080818 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 00:05:05.080818 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 00:05:05.080818 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 00:05:05.080818 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 00:05:05.081889 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:05:05.081889 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:05:05.081889 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 00:05:05.081889 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 00:05:05.081889 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 00:05:05.081889 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 00:05:05.081889 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:05:05.081889 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:05:05.081889 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:05:05.081889 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:05:05.081889 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:05:05.081889 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:05:05.081889 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:05:05.081889 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:05:05.081889 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:05:05.081889 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:05:05.081889 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:05:05.081889 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:05:05.083032 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:05:05.083032 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:05:05.083032 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 00:05:05.083032 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 00:05:05.083032 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 00:05:05.083032 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 00:05:05.083032 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 00:05:05.083032 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 00:05:05.083032 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 12 00:05:05.083032 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:05:05.083032 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 472/944] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:05:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:05:05.087745 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac14d0b-04f8-4cbd-bb66-459bfd29d5e1 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:05:05.148221 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:05:05.185175 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac14d0b-04f8-4cbd-bb66-459bfd29d5e1 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:05:05.216606 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab95d485-0499-4125-af07-d3abc209f1a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:05:05.217208 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab95d485-0499-4125-af07-d3abc209f1a9 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:05:05.217482 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab95d485-0499-4125-af07-d3abc209f1a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:05:05.217750 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab95d485-0499-4125-af07-d3abc209f1a9 None admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:05:05.218460 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab95d485-0499-4125-af07-d3abc209f1a9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:05:05.218761 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab95d485-0499-4125-af07-d3abc209f1a9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:05:05.220204 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab95d485-0499-4125-af07-d3abc209f1a9 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:05:05.237060 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 473/945] 10.222.0.36 () {56 vars in 1143 bytes} [Wed Oct 12 00:05:05 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:05:05.243441 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80a488a-e7f9-4386-8efc-1f4e1d57d36d None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:05:05.339875 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80a488a-e7f9-4386-8efc-1f4e1d57d36d None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:05:05.341351 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:05:05.373647 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafe3277-8daf-4500-a3a9-1ead1caaf304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:05:05.374271 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe3277-8daf-4500-a3a9-1ead1caaf304 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:05:05.374546 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe3277-8daf-4500-a3a9-1ead1caaf304 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:05:05.374855 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe3277-8daf-4500-a3a9-1ead1caaf304 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:05:05.384718 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafe3277-8daf-4500-a3a9-1ead1caaf304 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:05:05.386140 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafe3277-8daf-4500-a3a9-1ead1caaf304 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:05:05.400293 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-bafe3277-8daf-4500-a3a9-1ead1caaf304 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:05:05.400293 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:05:05.400293 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:05:05.400293 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:05:05.400293 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:05:05.400293 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:05:05.400293 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:05:05.400293 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:05:05.400293 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:05:05.400293 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:05:05.400293 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:05:05.400293 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:05:05.400293 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 00:05:05.400293 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 00:05:05.400293 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 00:05:05.400293 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 00:05:05.405886 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:05:05.405886 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:05:05.405886 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:05:05.405886 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:05:05.405886 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:05:05.405886 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:05:05.405886 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:05:05.405886 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:05:05.405886 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:05:05.405886 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:05:05.405886 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:05:05.405886 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:05:05.405886 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:05:05.405886 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:05:05.405886 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:05:05.405886 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:05:05.405886 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 00:05:05.406995 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 00:05:05.406995 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:05:05.406995 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:05:05.406995 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 473/946] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:05:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:05:05.407265 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab95d485-0499-4125-af07-d3abc209f1a9 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:05:05.508846 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab95d485-0499-4125-af07-d3abc209f1a9 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:05:05.528786 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:05:05.542891 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15da6ac-03fe-4c84-944e-24b66cf164db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:05:05.543592 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15da6ac-03fe-4c84-944e-24b66cf164db None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:05:05.543865 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15da6ac-03fe-4c84-944e-24b66cf164db None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:05:05.544129 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15da6ac-03fe-4c84-944e-24b66cf164db None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:05:05.544791 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15da6ac-03fe-4c84-944e-24b66cf164db None admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:05:05.545086 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15da6ac-03fe-4c84-944e-24b66cf164db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:05:05.546486 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15da6ac-03fe-4c84-944e-24b66cf164db None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:05:05.557978 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 474/947] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Oct 12 00:05:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:05:05.563646 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafe3277-8daf-4500-a3a9-1ead1caaf304 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:05:05.662341 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafe3277-8daf-4500-a3a9-1ead1caaf304 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:05:05.695117 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69d274a-6ca2-4981-ba91-4a14c587ab55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:05:05.695769 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69d274a-6ca2-4981-ba91-4a14c587ab55 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:05:05.696046 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69d274a-6ca2-4981-ba91-4a14c587ab55 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:05:05.696318 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69d274a-6ca2-4981-ba91-4a14c587ab55 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:05:05.703488 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69d274a-6ca2-4981-ba91-4a14c587ab55 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:05:05.704985 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69d274a-6ca2-4981-ba91-4a14c587ab55 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:05:05.716706 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-a69d274a-6ca2-4981-ba91-4a14c587ab55 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 00:05:05.716706 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:05:05.716706 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:05:05.716706 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:05:05.716706 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:05:05.716706 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:05:05.716706 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:05:05.716706 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:05:05.716706 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:05:05.716706 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:05:05.716706 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:05:05.716706 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:05:05.716706 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 00:05:05.716706 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 00:05:05.716706 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 00:05:05.717788 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 00:05:05.717788 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:05:05.717788 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:05:05.717788 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:05:05.717788 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:05:05.717788 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:05:05.717788 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:05:05.717788 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:05:05.717788 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:05:05.717788 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:05:05.717788 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:05:05.717788 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:05:05.717788 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:05:05.717788 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:05:05.717788 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:05:05.717788 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:05:05.717788 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:05:05.718874 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 00:05:05.718874 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 00:05:05.718874 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 00:05:05.718874 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 00:05:05.718874 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 00:05:05.718874 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 00:05:05.718874 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 00:05:05.718874 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:05:05.718874 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 474/948] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Oct 12 00:05:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:05:05.723644 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15da6ac-03fe-4c84-944e-24b66cf164db None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:05:05.821198 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15da6ac-03fe-4c84-944e-24b66cf164db None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:05:05.853431 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3950efea-8def-4f43-ac2b-42bd061fa1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:05:05.854101 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950efea-8def-4f43-ac2b-42bd061fa1d8 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:05:05.854371 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950efea-8def-4f43-ac2b-42bd061fa1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:05:05.854637 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950efea-8def-4f43-ac2b-42bd061fa1d8 None admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:05:05.855310 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3950efea-8def-4f43-ac2b-42bd061fa1d8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:05:05.855644 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3950efea-8def-4f43-ac2b-42bd061fa1d8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:05:05.857047 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3950efea-8def-4f43-ac2b-42bd061fa1d8 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:05:05.868368 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 475/949] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Oct 12 00:05:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:05:05.874540 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69d274a-6ca2-4981-ba91-4a14c587ab55 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:05:05.891680 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:05:05.974033 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69d274a-6ca2-4981-ba91-4a14c587ab55 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:05:06.008522 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6015eb-187d-44e3-85fa-ec2043d6b80c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:05:06.009161 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6015eb-187d-44e3-85fa-ec2043d6b80c None admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:05:06.009547 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6015eb-187d-44e3-85fa-ec2043d6b80c None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:05:06.009817 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6015eb-187d-44e3-85fa-ec2043d6b80c None admin] PATH_INFO: `/v3/projects/0697927c6aac43d4b14a750fbbb1145e/users/1e528da752ed4c78bc747215909f6569/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:05:06.011724 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6015eb-187d-44e3-85fa-ec2043d6b80c None admin] RBAC: Authorizing `identity:create_grant(project_id=0697927c6aac43d4b14a750fbbb1145e, user_id=1e528da752ed4c78bc747215909f6569, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:05:06.013147 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6015eb-187d-44e3-85fa-ec2043d6b80c None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:05:06.066055 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 475/950] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Oct 12 00:05:05 2022] PUT /identity/v3/projects/0697927c6aac43d4b14a750fbbb1145e/users/1e528da752ed4c78bc747215909f6569/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:05:08.198060 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2856d834-1f55-4d60-b1df-9a3331fe6c7a None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:05:08.198656 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2856d834-1f55-4d60-b1df-9a3331fe6c7a None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:05:08.198656 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2856d834-1f55-4d60-b1df-9a3331fe6c7a None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:05:08.199940 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 476/951] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:05:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:05:08.205085 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb919a4e-9aee-4f56-a807-0e0bb452bd65 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:05:08.205513 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb919a4e-9aee-4f56-a807-0e0bb452bd65 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:05:08.205620 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb919a4e-9aee-4f56-a807-0e0bb452bd65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:05:08.223486 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-cb919a4e-9aee-4f56-a807-0e0bb452bd65 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:05:08.224671 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-cb919a4e-9aee-4f56-a807-0e0bb452bd65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:05:08.291848 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 476/952] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:05:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 00:05:08.563507 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2856d834-1f55-4d60-b1df-9a3331fe6c7a None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:05:08.606837 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:05:08.663520 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2856d834-1f55-4d60-b1df-9a3331fe6c7a None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:05:08.697303 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9109782-7945-40d8-99ef-7a7cc45363fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:05:08.697924 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9109782-7945-40d8-99ef-7a7cc45363fd None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:05:08.698201 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9109782-7945-40d8-99ef-7a7cc45363fd None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:05:08.698469 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9109782-7945-40d8-99ef-7a7cc45363fd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:05:08.708317 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9109782-7945-40d8-99ef-7a7cc45363fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 00:05:08.713638 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-f9109782-7945-40d8-99ef-7a7cc45363fd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 00:05:08.713638 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:05:08.713638 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:05:08.713638 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:05:08.713638 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:05:08.713638 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:05:08.713638 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:05:08.713638 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:05:08.713638 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:05:08.713638 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:05:08.713638 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:05:08.713638 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:05:08.713638 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 00:05:08.713638 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 00:05:08.713638 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 00:05:08.713638 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 00:05:08.714925 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 00:05:08.714925 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 00:05:08.714925 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 00:05:08.714925 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 00:05:08.714925 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:05:08.714925 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:05:08.714925 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:05:08.714925 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:05:08.714925 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:05:08.714925 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:05:08.714925 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:05:08.714925 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:05:08.714925 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:05:08.714925 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:05:08.714925 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:05:08.714925 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:05:08.720605 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:05:08.720605 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:05:08.720605 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:05:08.720605 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:05:08.720605 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 00:05:08.720605 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 00:05:08.720605 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 00:05:08.720605 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 00:05:08.720605 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 00:05:08.720605 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 00:05:08.720605 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 00:05:08.720605 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:05:08.720605 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 477/953] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Oct 12 00:05:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:05:08.721602 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb919a4e-9aee-4f56-a807-0e0bb452bd65 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:05:08.723896 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:05:08.816551 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb919a4e-9aee-4f56-a807-0e0bb452bd65 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:05:08.849672 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696d1f04-83f6-4189-81c7-be6d501b49f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:05:08.850269 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696d1f04-83f6-4189-81c7-be6d501b49f6 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:05:08.850546 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696d1f04-83f6-4189-81c7-be6d501b49f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:05:08.850816 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696d1f04-83f6-4189-81c7-be6d501b49f6 None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:05:08.851500 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696d1f04-83f6-4189-81c7-be6d501b49f6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:05:08.851798 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696d1f04-83f6-4189-81c7-be6d501b49f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:05:08.853165 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696d1f04-83f6-4189-81c7-be6d501b49f6 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:05:08.860859 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 477/954] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Oct 12 00:05:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:05:08.866728 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9109782-7945-40d8-99ef-7a7cc45363fd None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:05:08.961794 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:05:08.964740 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9109782-7945-40d8-99ef-7a7cc45363fd None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:05:08.996617 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7217d4-7b4c-4af3-9d51-21c4b7abc3ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:05:08.997236 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7217d4-7b4c-4af3-9d51-21c4b7abc3ee None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:05:08.997509 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7217d4-7b4c-4af3-9d51-21c4b7abc3ee None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:05:08.997774 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7217d4-7b4c-4af3-9d51-21c4b7abc3ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:05:09.008231 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7217d4-7b4c-4af3-9d51-21c4b7abc3ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:05:09.014287 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7217d4-7b4c-4af3-9d51-21c4b7abc3ee None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:05:09.024825 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-1a7217d4-7b4c-4af3-9d51-21c4b7abc3ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:05:09.024825 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:05:09.024825 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:05:09.024825 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:05:09.024825 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:05:09.024825 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:05:09.024825 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:05:09.024825 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:05:09.024825 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:05:09.024825 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:05:09.024825 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:05:09.024825 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:05:09.024825 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 00:05:09.024825 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 00:05:09.024825 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 00:05:09.024825 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 00:05:09.025983 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:05:09.025983 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:05:09.025983 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:05:09.025983 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:05:09.025983 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:05:09.025983 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:05:09.025983 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:05:09.025983 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:05:09.025983 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:05:09.025983 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:05:09.025983 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:05:09.025983 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:05:09.025983 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:05:09.025983 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:05:09.025983 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:05:09.025983 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:05:09.025983 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 00:05:09.026927 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 00:05:09.026927 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:05:09.026927 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:05:09.026927 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 478/955] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:05:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:05:09.031771 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696d1f04-83f6-4189-81c7-be6d501b49f6 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:05:09.095488 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:05:09.128614 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696d1f04-83f6-4189-81c7-be6d501b49f6 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:05:09.162261 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57655863-7f1d-46be-8682-dfed1ed1cf58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:05:09.162876 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57655863-7f1d-46be-8682-dfed1ed1cf58 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:05:09.163150 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57655863-7f1d-46be-8682-dfed1ed1cf58 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:05:09.163446 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57655863-7f1d-46be-8682-dfed1ed1cf58 None admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:05:09.164072 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57655863-7f1d-46be-8682-dfed1ed1cf58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:05:09.164368 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57655863-7f1d-46be-8682-dfed1ed1cf58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:05:09.165730 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57655863-7f1d-46be-8682-dfed1ed1cf58 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:05:09.176999 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 478/956] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Oct 12 00:05:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:05:09.183217 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7217d4-7b4c-4af3-9d51-21c4b7abc3ee None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:05:09.281183 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7217d4-7b4c-4af3-9d51-21c4b7abc3ee None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:05:09.313697 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5266a0e5-564e-4352-9774-29be7f146725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:05:09.314309 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5266a0e5-564e-4352-9774-29be7f146725 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:05:09.314582 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5266a0e5-564e-4352-9774-29be7f146725 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:05:09.314849 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5266a0e5-564e-4352-9774-29be7f146725 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:05:09.322185 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5266a0e5-564e-4352-9774-29be7f146725 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:05:09.327117 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5266a0e5-564e-4352-9774-29be7f146725 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:05:09.335064 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-5266a0e5-564e-4352-9774-29be7f146725 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 12 00:05:09.335064 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:05:09.335064 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:05:09.335064 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:05:09.335064 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:05:09.335064 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:05:09.335064 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:05:09.335064 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:05:09.335064 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:05:09.335064 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:05:09.335064 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:05:09.335064 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:05:09.335064 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 00:05:09.335064 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 00:05:09.335064 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 00:05:09.335064 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 00:05:09.336331 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:05:09.336331 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:05:09.336331 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 00:05:09.336331 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 00:05:09.336331 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 00:05:09.336331 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 00:05:09.336331 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:05:09.336331 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:05:09.336331 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:05:09.336331 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:05:09.336331 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:05:09.336331 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:05:09.336331 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:05:09.336331 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:05:09.336331 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:05:09.336331 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:05:09.336331 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:05:09.336331 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:05:09.337479 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:05:09.337479 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:05:09.337479 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 00:05:09.337479 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 00:05:09.337479 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 00:05:09.337479 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 00:05:09.337479 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 00:05:09.337479 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 00:05:09.337479 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 12 00:05:09.337479 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:05:09.337479 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 479/957] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:05:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:05:09.341927 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57655863-7f1d-46be-8682-dfed1ed1cf58 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:05:09.437540 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57655863-7f1d-46be-8682-dfed1ed1cf58 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:05:09.462050 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:05:09.470361 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22aee57-0bd5-4cd2-b478-b0818e4f31ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:05:09.470932 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22aee57-0bd5-4cd2-b478-b0818e4f31ea None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:05:09.471209 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22aee57-0bd5-4cd2-b478-b0818e4f31ea None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:05:09.471499 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22aee57-0bd5-4cd2-b478-b0818e4f31ea None admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:05:09.472211 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22aee57-0bd5-4cd2-b478-b0818e4f31ea None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:05:09.472510 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22aee57-0bd5-4cd2-b478-b0818e4f31ea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:05:09.473925 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22aee57-0bd5-4cd2-b478-b0818e4f31ea None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:05:09.490877 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 479/958] 10.222.0.36 () {56 vars in 1143 bytes} [Wed Oct 12 00:05:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:05:09.496688 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5266a0e5-564e-4352-9774-29be7f146725 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:05:09.512391 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:05:09.595066 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5266a0e5-564e-4352-9774-29be7f146725 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:05:09.629503 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b946ad8-d4f7-43fa-8e8f-8db0631630d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:05:09.630098 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b946ad8-d4f7-43fa-8e8f-8db0631630d0 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:05:09.630415 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b946ad8-d4f7-43fa-8e8f-8db0631630d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:05:09.630732 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b946ad8-d4f7-43fa-8e8f-8db0631630d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:05:09.695791 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b946ad8-d4f7-43fa-8e8f-8db0631630d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:05:09.697167 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b946ad8-d4f7-43fa-8e8f-8db0631630d0 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:05:09.710176 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:05:09.791203 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 480/959] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Oct 12 00:05:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 12 00:05:09.797242 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22aee57-0bd5-4cd2-b478-b0818e4f31ea None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:05:09.894143 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22aee57-0bd5-4cd2-b478-b0818e4f31ea None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:05:09.926952 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f8023e-b578-43be-8474-3c7e6701db02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:05:09.927593 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f8023e-b578-43be-8474-3c7e6701db02 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:05:09.928467 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f8023e-b578-43be-8474-3c7e6701db02 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:05:09.928748 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f8023e-b578-43be-8474-3c7e6701db02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:05:09.938405 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f8023e-b578-43be-8474-3c7e6701db02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:05:09.939909 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f8023e-b578-43be-8474-3c7e6701db02 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:05:09.954253 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-18f8023e-b578-43be-8474-3c7e6701db02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:05:09.954253 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:05:09.954253 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:05:09.954253 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:05:09.954253 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:05:09.954253 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:05:09.954253 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:05:09.954253 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:05:09.954253 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:05:09.954253 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:05:09.954253 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:05:09.954253 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:05:09.954253 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 00:05:09.954253 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 00:05:09.954253 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 00:05:09.954253 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 00:05:09.955422 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:05:09.955422 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:05:09.955422 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:05:09.955422 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:05:09.955422 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:05:09.955422 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:05:09.955422 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:05:09.955422 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:05:09.955422 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:05:09.955422 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:05:09.955422 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:05:09.955422 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:05:09.955422 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:05:09.955422 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:05:09.955422 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:05:09.955422 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:05:09.955422 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 00:05:09.956536 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 00:05:09.956536 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 00:05:09.956536 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:05:09.956536 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 480/960] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:05:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:05:09.961738 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b946ad8-d4f7-43fa-8e8f-8db0631630d0 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:05:10.061430 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b946ad8-d4f7-43fa-8e8f-8db0631630d0 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:05:10.073729 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:05:10.094231 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197384d9-dd82-4b85-9577-9c262d7535cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:05:10.094837 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197384d9-dd82-4b85-9577-9c262d7535cb None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:05:10.095113 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197384d9-dd82-4b85-9577-9c262d7535cb None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:05:10.095403 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197384d9-dd82-4b85-9577-9c262d7535cb None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:05:10.096034 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197384d9-dd82-4b85-9577-9c262d7535cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:05:10.096331 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197384d9-dd82-4b85-9577-9c262d7535cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:05:10.097753 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197384d9-dd82-4b85-9577-9c262d7535cb None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:05:10.109829 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 481/961] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Oct 12 00:05:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:05:10.115760 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f8023e-b578-43be-8474-3c7e6701db02 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:05:10.150136 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:05:10.215120 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f8023e-b578-43be-8474-3c7e6701db02 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:05:10.248835 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d672ee54-2700-4341-a298-efc138e483d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:05:10.249464 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d672ee54-2700-4341-a298-efc138e483d5 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:05:10.249753 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d672ee54-2700-4341-a298-efc138e483d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:05:10.250022 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d672ee54-2700-4341-a298-efc138e483d5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:05:10.257108 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d672ee54-2700-4341-a298-efc138e483d5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:05:10.259802 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d672ee54-2700-4341-a298-efc138e483d5 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:05:10.270009 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-d672ee54-2700-4341-a298-efc138e483d5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 00:05:10.270009 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:05:10.270009 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:05:10.270009 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:05:10.270009 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:05:10.270009 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:05:10.270009 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:05:10.270009 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:05:10.270009 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:05:10.270009 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:05:10.270009 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:05:10.270009 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:05:10.270009 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 00:05:10.270009 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 00:05:10.270009 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 00:05:10.271014 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 00:05:10.271014 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:05:10.271014 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:05:10.271014 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:05:10.271014 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:05:10.271014 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:05:10.271014 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:05:10.271014 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:05:10.271014 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Oct 12 00:05:10.271014 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:05:10.271014 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:05:10.271014 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:05:10.271014 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:05:10.271014 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:05:10.271014 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:05:10.271014 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:05:10.271014 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:05:10.279790 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 00:05:10.279790 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 00:05:10.279790 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 00:05:10.279790 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 00:05:10.279790 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 00:05:10.279790 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 00:05:10.279790 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 00:05:10.279790 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Oct 12 00:05:10.279790 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 481/962] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Oct 12 00:05:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:05:10.280422 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197384d9-dd82-4b85-9577-9c262d7535cb None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:05:10.373050 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197384d9-dd82-4b85-9577-9c262d7535cb None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:05:10.407608 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c08d76-1424-46ed-845c-691909dd6a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:05:10.408215 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c08d76-1424-46ed-845c-691909dd6a51 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:05:10.408522 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c08d76-1424-46ed-845c-691909dd6a51 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:05:10.408794 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c08d76-1424-46ed-845c-691909dd6a51 None admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:05:10.409475 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c08d76-1424-46ed-845c-691909dd6a51 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:05:10.409775 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c08d76-1424-46ed-845c-691909dd6a51 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:05:10.411245 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c08d76-1424-46ed-845c-691909dd6a51 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:05:10.422367 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 482/963] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Oct 12 00:05:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:05:10.428428 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d672ee54-2700-4341-a298-efc138e483d5 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:05:10.520740 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:05:10.526753 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d672ee54-2700-4341-a298-efc138e483d5 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:05:10.559065 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6e23e7-8c8e-4632-beb7-128a686b0ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:05:10.559696 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6e23e7-8c8e-4632-beb7-128a686b0ca7 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:05:10.559970 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6e23e7-8c8e-4632-beb7-128a686b0ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:05:10.560236 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6e23e7-8c8e-4632-beb7-128a686b0ca7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:05:10.560945 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6e23e7-8c8e-4632-beb7-128a686b0ca7 None admin] RBAC: Adding query filter params (role.id=b2e41f0d7a304dc7af8b6a232049b6a2, scope.project.id=0697927c6aac43d4b14a750fbbb1145e, user.id=1e528da752ed4c78bc747215909f6569) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:05:10.561246 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6e23e7-8c8e-4632-beb7-128a686b0ca7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:05:10.562664 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6e23e7-8c8e-4632-beb7-128a686b0ca7 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:05:10.568190 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 482/964] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Oct 12 00:05:10 2022] GET /identity/v3/role_assignments?user.id=1e528da752ed4c78bc747215909f6569&scope.project.id=0697927c6aac43d4b14a750fbbb1145e&role.id=b2e41f0d7a304dc7af8b6a232049b6a2 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:05:12.700692 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f0112e-d6ec-437c-9a50-d396d6b1ceef None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:05:12.701217 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f0112e-d6ec-437c-9a50-d396d6b1ceef None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:05:12.701217 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f0112e-d6ec-437c-9a50-d396d6b1ceef None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:05:12.702565 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 483/965] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:05:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 00:05:12.707885 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a209cf2c-1945-415b-89b9-7cbf0e4ea96a None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:05:12.708264 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a209cf2c-1945-415b-89b9-7cbf0e4ea96a None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:05:12.708423 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a209cf2c-1945-415b-89b9-7cbf0e4ea96a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:05:12.726875 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-a209cf2c-1945-415b-89b9-7cbf0e4ea96a None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:05:12.728174 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-a209cf2c-1945-415b-89b9-7cbf0e4ea96a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:05:12.792694 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 483/966] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:05:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 00:05:13.065639 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f0112e-d6ec-437c-9a50-d396d6b1ceef None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:05:13.096372 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:05:13.165315 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f0112e-d6ec-437c-9a50-d396d6b1ceef None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:05:13.198682 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf120c6-78ac-476e-bdbe-e2f876bce6ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:05:13.199303 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf120c6-78ac-476e-bdbe-e2f876bce6ad None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:05:13.199605 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf120c6-78ac-476e-bdbe-e2f876bce6ad None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:05:13.199875 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf120c6-78ac-476e-bdbe-e2f876bce6ad None admin] PATH_INFO: `/v3/services/placement` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:05:13.200450 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf120c6-78ac-476e-bdbe-e2f876bce6ad None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:05:13.201809 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf120c6-78ac-476e-bdbe-e2f876bce6ad None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:05:13.210927 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-6bf120c6-78ac-476e-bdbe-e2f876bce6ad None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 12 00:05:13.210927 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:05:13.210927 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 00:05:13.210927 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 00:05:13.210927 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 00:05:13.210927 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 00:05:13.210927 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 00:05:13.210927 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 00:05:13.210927 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 12 00:05:13.210927 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:05:13.210927 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 00:05:13.210927 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:05:13.210927 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:05:13.210927 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:05:13.210927 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:05:13.210927 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:05:13.210927 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:05:13.217566 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:05:13.217566 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:05:13.217566 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:05:13.217566 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:05:13.217566 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:05:13.217566 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:05:13.217566 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 00:05:13.217566 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 00:05:13.217566 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 00:05:13.217566 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 00:05:13.217566 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:05:13.217566 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:05:13.217566 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:05:13.217566 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:05:13.217566 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:05:13.217566 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:05:13.218686 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:05:13.218686 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:05:13.218686 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:05:13.218686 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:05:13.218686 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:05:13.218686 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:05:13.218686 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:05:13.218686 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:05:13.218686 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:05:13.218686 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:05:13.218686 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 00:05:13.218686 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 00:05:13.218686 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 12 00:05:13.218686 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:05:13.218686 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 484/967] 10.222.0.36 () {56 vars in 1117 bytes} [Wed Oct 12 00:05:13 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:05:13.219697 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a209cf2c-1945-415b-89b9-7cbf0e4ea96a None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:05:13.307177 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:05:13.316462 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a209cf2c-1945-415b-89b9-7cbf0e4ea96a None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:05:13.349920 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b69f2e0-ad8c-4985-83dd-95ea438fd382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:05:13.350522 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b69f2e0-ad8c-4985-83dd-95ea438fd382 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:05:13.350796 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b69f2e0-ad8c-4985-83dd-95ea438fd382 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:05:13.351062 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b69f2e0-ad8c-4985-83dd-95ea438fd382 None admin] PATH_INFO: `/v3/services` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:05:13.351714 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b69f2e0-ad8c-4985-83dd-95ea438fd382 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:05:13.352016 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b69f2e0-ad8c-4985-83dd-95ea438fd382 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:05:13.353375 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b69f2e0-ad8c-4985-83dd-95ea438fd382 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:05:13.358263 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 484/968] 10.222.0.36 () {56 vars in 1116 bytes} [Wed Oct 12 00:05:13 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:05:13.364064 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf120c6-78ac-476e-bdbe-e2f876bce6ad None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:05:13.455776 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:05:13.461814 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf120c6-78ac-476e-bdbe-e2f876bce6ad None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:05:13.493955 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b70bb16-dc41-4eab-9172-c788d14aa4bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:05:13.494550 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b70bb16-dc41-4eab-9172-c788d14aa4bd None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:05:13.494821 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b70bb16-dc41-4eab-9172-c788d14aa4bd None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:05:13.495090 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b70bb16-dc41-4eab-9172-c788d14aa4bd None admin] PATH_INFO: `/v3/services` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:05:13.495738 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b70bb16-dc41-4eab-9172-c788d14aa4bd None admin] RBAC: Adding query filter params (type=placement) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:05:13.496033 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b70bb16-dc41-4eab-9172-c788d14aa4bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:05:13.497480 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b70bb16-dc41-4eab-9172-c788d14aa4bd None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:05:13.501713 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 485/969] 10.222.0.36 () {56 vars in 1116 bytes} [Wed Oct 12 00:05:13 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:05:15.641564 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51109941-1acd-4b31-88d1-6d0babad3eab None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:05:15.642089 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51109941-1acd-4b31-88d1-6d0babad3eab None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:05:15.642089 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51109941-1acd-4b31-88d1-6d0babad3eab None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:05:15.643447 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 485/970] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:05:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:05:15.649070 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67737f18-6e17-4993-bb0a-fff411641778 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:05:15.649527 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67737f18-6e17-4993-bb0a-fff411641778 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:05:15.649656 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67737f18-6e17-4993-bb0a-fff411641778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:05:15.668116 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-67737f18-6e17-4993-bb0a-fff411641778 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:05:15.669588 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-67737f18-6e17-4993-bb0a-fff411641778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:05:15.734330 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 486/971] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:05:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 00:05:16.002496 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51109941-1acd-4b31-88d1-6d0babad3eab None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:05:16.103670 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51109941-1acd-4b31-88d1-6d0babad3eab None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:05:16.136779 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02544b86-7574-44e4-887b-e0f0b24ae85b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:05:16.137394 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02544b86-7574-44e4-887b-e0f0b24ae85b None admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:05:16.137670 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02544b86-7574-44e4-887b-e0f0b24ae85b None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:05:16.137938 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02544b86-7574-44e4-887b-e0f0b24ae85b None admin] PATH_INFO: `/v3/services` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:05:16.138594 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02544b86-7574-44e4-887b-e0f0b24ae85b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:05:16.139982 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02544b86-7574-44e4-887b-e0f0b24ae85b None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:05:16.218201 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 486/972] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Oct 12 00:05:16 2022] POST /identity/v3/services => generated 246 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:05:18.371960 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47891a0-9579-4983-93f6-fd5298173f2f None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:05:18.372523 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47891a0-9579-4983-93f6-fd5298173f2f None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:05:18.372523 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47891a0-9579-4983-93f6-fd5298173f2f None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:05:18.373823 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 487/973] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:05:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:05:18.379347 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb29617-d745-4f57-a1c5-1e99fc617d78 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:05:18.381149 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb29617-d745-4f57-a1c5-1e99fc617d78 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:05:18.381149 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb29617-d745-4f57-a1c5-1e99fc617d78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:05:18.398543 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-6fb29617-d745-4f57-a1c5-1e99fc617d78 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:05:18.399811 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-6fb29617-d745-4f57-a1c5-1e99fc617d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:05:18.408937 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:05:18.471346 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 487/974] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:05:18 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 00:05:18.744489 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47891a0-9579-4983-93f6-fd5298173f2f None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:05:18.832431 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:05:18.842663 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47891a0-9579-4983-93f6-fd5298173f2f None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:05:18.876632 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0518b4ef-1531-40a2-8125-3cd17e4140db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:05:18.877246 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0518b4ef-1531-40a2-8125-3cd17e4140db None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:05:18.877518 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0518b4ef-1531-40a2-8125-3cd17e4140db None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:05:18.877782 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0518b4ef-1531-40a2-8125-3cd17e4140db None admin] PATH_INFO: `/v3/services/placement` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:05:18.878350 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0518b4ef-1531-40a2-8125-3cd17e4140db None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:05:18.879990 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0518b4ef-1531-40a2-8125-3cd17e4140db None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:05:18.888523 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-0518b4ef-1531-40a2-8125-3cd17e4140db None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 12 00:05:18.888523 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:05:18.888523 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 00:05:18.888523 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 00:05:18.888523 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 00:05:18.888523 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 00:05:18.888523 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 00:05:18.888523 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 00:05:18.888523 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 12 00:05:18.888523 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:05:18.888523 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 00:05:18.888523 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:05:18.888523 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:05:18.888523 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:05:18.888523 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:05:18.888523 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:05:18.888523 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:05:18.889553 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:05:18.889553 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:05:18.889553 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:05:18.889553 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:05:18.889553 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:05:18.889553 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:05:18.889553 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 00:05:18.889553 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 00:05:18.889553 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 00:05:18.889553 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 00:05:18.889553 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:05:18.889553 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:05:18.889553 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:05:18.889553 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:05:18.889553 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:05:18.889553 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:05:18.890389 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:05:18.890389 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:05:18.890389 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:05:18.890389 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:05:18.890389 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:05:18.890389 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:05:18.890389 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:05:18.890389 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:05:18.890389 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:05:18.890389 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:05:18.890389 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 00:05:18.890389 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 00:05:18.890389 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 12 00:05:18.890389 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:05:18.890389 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 488/975] 10.222.0.36 () {56 vars in 1117 bytes} [Wed Oct 12 00:05:18 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:05:18.896053 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb29617-d745-4f57-a1c5-1e99fc617d78 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:05:18.991356 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb29617-d745-4f57-a1c5-1e99fc617d78 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:05:19.025871 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbb3ac2-4041-4165-9ed4-a6f9893ce5e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:05:19.026416 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbb3ac2-4041-4165-9ed4-a6f9893ce5e0 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:05:19.026695 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbb3ac2-4041-4165-9ed4-a6f9893ce5e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:05:19.026959 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbb3ac2-4041-4165-9ed4-a6f9893ce5e0 None admin] PATH_INFO: `/v3/services` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:05:19.027617 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbb3ac2-4041-4165-9ed4-a6f9893ce5e0 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:05:19.027914 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbb3ac2-4041-4165-9ed4-a6f9893ce5e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:05:19.029300 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbb3ac2-4041-4165-9ed4-a6f9893ce5e0 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:05:19.034346 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 488/976] 10.222.0.36 () {56 vars in 1116 bytes} [Wed Oct 12 00:05:18 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:05:19.041141 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0518b4ef-1531-40a2-8125-3cd17e4140db None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:05:19.139147 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0518b4ef-1531-40a2-8125-3cd17e4140db None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:05:19.172736 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf61b98b-0bd0-49f3-aa9d-95ec419a72fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:05:19.173336 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf61b98b-0bd0-49f3-aa9d-95ec419a72fb None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:05:19.173612 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf61b98b-0bd0-49f3-aa9d-95ec419a72fb None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:05:19.173879 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf61b98b-0bd0-49f3-aa9d-95ec419a72fb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:05:19.174564 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf61b98b-0bd0-49f3-aa9d-95ec419a72fb None admin] RBAC: Adding query filter params (interface=public, service_id=e79f52e22822426b9ced305b0f884fbc, region_id=RegionOne) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:05:19.174864 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf61b98b-0bd0-49f3-aa9d-95ec419a72fb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:05:19.176243 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf61b98b-0bd0-49f3-aa9d-95ec419a72fb None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:05:19.181185 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 489/977] 10.222.0.36 () {56 vars in 1251 bytes} [Wed Oct 12 00:05:19 2022] GET /identity/v3/endpoints?service_id=e79f52e22822426b9ced305b0f884fbc&interface=public®ion_id=RegionOne => generated 194 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:05:19.187058 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbb3ac2-4041-4165-9ed4-a6f9893ce5e0 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:05:19.190858 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:05:19.283572 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbb3ac2-4041-4165-9ed4-a6f9893ce5e0 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:05:19.317084 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060c01c8-cf98-4e5e-b62c-819fa350d2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:05:19.317688 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060c01c8-cf98-4e5e-b62c-819fa350d2f0 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:05:19.318482 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060c01c8-cf98-4e5e-b62c-819fa350d2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:05:19.318754 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060c01c8-cf98-4e5e-b62c-819fa350d2f0 None admin] PATH_INFO: `/v3/services` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:05:19.319351 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060c01c8-cf98-4e5e-b62c-819fa350d2f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:05:19.320742 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060c01c8-cf98-4e5e-b62c-819fa350d2f0 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:05:19.327353 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 489/978] 10.222.0.36 () {56 vars in 1087 bytes} [Wed Oct 12 00:05:19 2022] GET /identity/v3/services => generated 2191 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:05:21.512064 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b950f14-4a37-4741-9c2a-8076d9d7d76f None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:05:21.512631 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b950f14-4a37-4741-9c2a-8076d9d7d76f None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:05:21.512631 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b950f14-4a37-4741-9c2a-8076d9d7d76f None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:05:21.513962 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 490/979] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:05:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:05:21.519208 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aa3740-42ef-4a59-b0e5-9c10b7c045b5 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:05:21.519657 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aa3740-42ef-4a59-b0e5-9c10b7c045b5 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:05:21.519801 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aa3740-42ef-4a59-b0e5-9c10b7c045b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:05:21.538896 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-24aa3740-42ef-4a59-b0e5-9c10b7c045b5 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:05:21.540108 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-24aa3740-42ef-4a59-b0e5-9c10b7c045b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:05:21.601089 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:05:21.606446 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 490/980] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:05:21 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 00:05:21.877874 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b950f14-4a37-4741-9c2a-8076d9d7d76f None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:05:21.898617 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:05:21.978379 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b950f14-4a37-4741-9c2a-8076d9d7d76f None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:05:22.011937 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849d89e3-e45c-4b51-992a-205d0c9b9c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:05:22.012555 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849d89e3-e45c-4b51-992a-205d0c9b9c8e None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:05:22.012829 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849d89e3-e45c-4b51-992a-205d0c9b9c8e None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:05:22.013098 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849d89e3-e45c-4b51-992a-205d0c9b9c8e None admin] PATH_INFO: `/v3/services/placement` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:05:22.013667 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849d89e3-e45c-4b51-992a-205d0c9b9c8e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:05:22.015019 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849d89e3-e45c-4b51-992a-205d0c9b9c8e None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:05:22.023577 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-849d89e3-e45c-4b51-992a-205d0c9b9c8e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 12 00:05:22.023577 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:05:22.023577 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 00:05:22.023577 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 00:05:22.023577 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 00:05:22.023577 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 00:05:22.023577 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 00:05:22.023577 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 00:05:22.023577 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 12 00:05:22.023577 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:05:22.023577 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 00:05:22.023577 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:05:22.023577 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 00:05:22.023577 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 00:05:22.023577 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 00:05:22.023577 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 00:05:22.023577 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 00:05:22.024789 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 00:05:22.024789 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 00:05:22.024789 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 00:05:22.024789 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 00:05:22.024789 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 00:05:22.024789 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 00:05:22.024789 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 00:05:22.024789 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 00:05:22.024789 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 00:05:22.024789 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 00:05:22.024789 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 00:05:22.024789 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 00:05:22.024789 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 00:05:22.024789 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 00:05:22.024789 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 00:05:22.024789 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 00:05:22.025794 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 00:05:22.025794 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Oct 12 00:05:22.025794 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 00:05:22.025794 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Oct 12 00:05:22.025794 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 00:05:22.025794 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 00:05:22.025794 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 00:05:22.025794 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Oct 12 00:05:22.025794 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 00:05:22.025794 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Oct 12 00:05:22.025794 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 00:05:22.025794 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 00:05:22.025794 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 12 00:05:22.025794 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Oct 12 00:05:22.025794 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 491/981] 10.222.0.36 () {56 vars in 1117 bytes} [Wed Oct 12 00:05:21 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 00:05:22.030642 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aa3740-42ef-4a59-b0e5-9c10b7c045b5 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:05:22.126366 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aa3740-42ef-4a59-b0e5-9c10b7c045b5 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:05:22.167104 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cc7a06-f037-48da-872f-7659726930e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:05:22.167104 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cc7a06-f037-48da-872f-7659726930e6 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:05:22.167104 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cc7a06-f037-48da-872f-7659726930e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:05:22.167104 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cc7a06-f037-48da-872f-7659726930e6 None admin] PATH_INFO: `/v3/services` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:05:22.167528 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cc7a06-f037-48da-872f-7659726930e6 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:05:22.167528 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cc7a06-f037-48da-872f-7659726930e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:05:22.167528 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cc7a06-f037-48da-872f-7659726930e6 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:05:22.167817 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 491/982] 10.222.0.36 () {56 vars in 1116 bytes} [Wed Oct 12 00:05:22 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:05:22.173465 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849d89e3-e45c-4b51-992a-205d0c9b9c8e None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:05:22.253521 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:05:22.271618 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849d89e3-e45c-4b51-992a-205d0c9b9c8e None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:05:22.305035 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f037dd46-0edf-42ba-8f85-2f33e1a593ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:05:22.305624 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f037dd46-0edf-42ba-8f85-2f33e1a593ce None admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:05:22.305897 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f037dd46-0edf-42ba-8f85-2f33e1a593ce None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:05:22.306164 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f037dd46-0edf-42ba-8f85-2f33e1a593ce None admin] PATH_INFO: `/v3/endpoints` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:05:22.306815 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f037dd46-0edf-42ba-8f85-2f33e1a593ce None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:05:22.308201 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f037dd46-0edf-42ba-8f85-2f33e1a593ce None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:05:22.396824 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 492/983] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:05:22 2022] POST /identity/v3/endpoints => generated 331 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:05:32.164811 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=69 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:05:32.392365 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:11:31.477670 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25b2d6d-5f66-4c26-a5fa-f197508c2711 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:11:31.478259 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25b2d6d-5f66-4c26-a5fa-f197508c2711 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:11:31.478259 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25b2d6d-5f66-4c26-a5fa-f197508c2711 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:11:31.479975 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 492/984] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:11:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:11:31.488617 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173596b8-b1df-4867-9df7-351a6d352f2a None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:11:31.488617 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173596b8-b1df-4867-9df7-351a6d352f2a None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:11:31.488617 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173596b8-b1df-4867-9df7-351a6d352f2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:11:31.576954 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-173596b8-b1df-4867-9df7-351a6d352f2a None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:11:31.578149 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-173596b8-b1df-4867-9df7-351a6d352f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:11:31.691507 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 493/985] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:11:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:11:31.988127 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4e0ed9-5cbe-4283-99c9-6a4c423a893c None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:11:31.988587 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4e0ed9-5cbe-4283-99c9-6a4c423a893c None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:11:31.988964 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4e0ed9-5cbe-4283-99c9-6a4c423a893c None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:11:31.990450 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 493/986] 10.222.0.36 () {54 vars in 964 bytes} [Wed Oct 12 00:11:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:11:31.996468 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a2d44a-55eb-4ee7-b9d2-94734b489cb7 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:11:31.996904 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a2d44a-55eb-4ee7-b9d2-94734b489cb7 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:11:31.997281 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a2d44a-55eb-4ee7-b9d2-94734b489cb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:11:32.080912 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-98a2d44a-55eb-4ee7-b9d2-94734b489cb7 None None] MFA Rules not processed for user `51215dab618141b4991e232724094140`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:11:32.082336 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-98a2d44a-55eb-4ee7-b9d2-94734b489cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:11:32.105800 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:11:32.171896 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 494/987] 10.222.0.36 () {58 vars in 1061 bytes} [Wed Oct 12 00:11:31 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:11:32.181846 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4e0ed9-5cbe-4283-99c9-6a4c423a893c None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:11:32.183813 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-ee4e0ed9-5cbe-4283-99c9-6a4c423a893c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:11:32.303237 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4e0ed9-5cbe-4283-99c9-6a4c423a893c None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:11:32.381051 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1a3c52-bbb4-4928-a02a-5b27eec563e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:11:32.381841 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1a3c52-bbb4-4928-a02a-5b27eec563e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:11:32.382225 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1a3c52-bbb4-4928-a02a-5b27eec563e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:11:32.382629 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1a3c52-bbb4-4928-a02a-5b27eec563e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:11:32.384393 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-ab1a3c52-bbb4-4928-a02a-5b27eec563e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:11:32.446387 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:11:32.510748 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1a3c52-bbb4-4928-a02a-5b27eec563e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:11:32.513204 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1a3c52-bbb4-4928-a02a-5b27eec563e3 service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:11:32.533954 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 494/988] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:11:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:11:34.957001 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba6a255-93e3-4cc5-9ad5-819d29f04dc3 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:11:34.957526 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba6a255-93e3-4cc5-9ad5-819d29f04dc3 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:11:34.957526 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba6a255-93e3-4cc5-9ad5-819d29f04dc3 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:11:34.958919 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 495/989] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:11:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 00:11:34.964191 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47628c65-5c71-4a31-b79e-ee2b77381155 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:11:34.964647 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47628c65-5c71-4a31-b79e-ee2b77381155 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:11:34.964749 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47628c65-5c71-4a31-b79e-ee2b77381155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:11:35.035080 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-47628c65-5c71-4a31-b79e-ee2b77381155 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:11:35.036229 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-47628c65-5c71-4a31-b79e-ee2b77381155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:11:35.129289 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 495/990] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:11:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:11:35.410054 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba6a255-93e3-4cc5-9ad5-819d29f04dc3 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:11:35.411637 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-9ba6a255-93e3-4cc5-9ad5-819d29f04dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:11:35.512797 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba6a255-93e3-4cc5-9ad5-819d29f04dc3 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:11:35.584103 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:11:35.586085 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b58df0-988b-4afe-8627-e698976afef2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:11:35.586683 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b58df0-988b-4afe-8627-e698976afef2 service neutron] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:11:35.586958 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b58df0-988b-4afe-8627-e698976afef2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:11:35.587224 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b58df0-988b-4afe-8627-e698976afef2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:11:35.588850 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-e8b58df0-988b-4afe-8627-e698976afef2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:11:35.700665 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b58df0-988b-4afe-8627-e698976afef2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:11:35.702008 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b58df0-988b-4afe-8627-e698976afef2 service neutron] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:11:35.720301 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 496/991] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:11:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:11:38.333493 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d960db01-ead3-4004-bb26-de5c72ec41fe None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:11:38.334028 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d960db01-ead3-4004-bb26-de5c72ec41fe None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:11:38.334028 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d960db01-ead3-4004-bb26-de5c72ec41fe None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:11:38.335385 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 496/992] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:11:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:11:38.340493 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5608b607-55ac-42cb-911d-7b45525c5982 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:11:38.340891 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5608b607-55ac-42cb-911d-7b45525c5982 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:11:38.341039 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5608b607-55ac-42cb-911d-7b45525c5982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:11:38.405718 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-5608b607-55ac-42cb-911d-7b45525c5982 None None] MFA Rules not processed for user `216d4237a0a44e7eaf170e6dcd700023`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:11:38.407558 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-5608b607-55ac-42cb-911d-7b45525c5982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:11:38.457188 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:11:38.500979 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 497/993] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:11:38 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:11:38.782871 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d960db01-ead3-4004-bb26-de5c72ec41fe None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:11:38.784473 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-d960db01-ead3-4004-bb26-de5c72ec41fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:11:38.794115 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:11:38.885515 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d960db01-ead3-4004-bb26-de5c72ec41fe None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:11:38.956291 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65504986-38ff-48c0-b046-ed11e3c9cf1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:11:38.956889 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65504986-38ff-48c0-b046-ed11e3c9cf1b service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:11:38.957161 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65504986-38ff-48c0-b046-ed11e3c9cf1b service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:11:38.957426 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65504986-38ff-48c0-b046-ed11e3c9cf1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:11:38.958993 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-65504986-38ff-48c0-b046-ed11e3c9cf1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:11:39.019449 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:11:39.065267 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65504986-38ff-48c0-b046-ed11e3c9cf1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:11:39.066710 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65504986-38ff-48c0-b046-ed11e3c9cf1b service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:11:39.084615 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 497/994] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:11:38 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:11:41.926360 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b16e61b-8c17-4acd-875b-83c7ed0a8ade None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:11:41.926925 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b16e61b-8c17-4acd-875b-83c7ed0a8ade None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:11:41.926925 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b16e61b-8c17-4acd-875b-83c7ed0a8ade None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:11:41.928277 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 498/995] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:11:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:11:41.933536 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5729877a-8a38-4783-b17d-12d5af07346b None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:11:41.933924 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5729877a-8a38-4783-b17d-12d5af07346b None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:11:41.934075 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5729877a-8a38-4783-b17d-12d5af07346b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:11:41.999464 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-5729877a-8a38-4783-b17d-12d5af07346b None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:11:42.001592 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-5729877a-8a38-4783-b17d-12d5af07346b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:11:42.094325 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 498/996] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:11:41 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:11:42.377114 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b16e61b-8c17-4acd-875b-83c7ed0a8ade None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:11:42.378662 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-9b16e61b-8c17-4acd-875b-83c7ed0a8ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:11:42.479806 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b16e61b-8c17-4acd-875b-83c7ed0a8ade None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:11:42.552951 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d6fd18-6c1f-4818-948c-d08a65ae12c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:11:42.553558 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6fd18-6c1f-4818-948c-d08a65ae12c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:11:42.553835 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6fd18-6c1f-4818-948c-d08a65ae12c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:11:42.554105 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6fd18-6c1f-4818-948c-d08a65ae12c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:11:42.555723 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-88d6fd18-6c1f-4818-948c-d08a65ae12c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:11:42.562625 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:11:42.663651 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d6fd18-6c1f-4818-948c-d08a65ae12c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:11:42.664989 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d6fd18-6c1f-4818-948c-d08a65ae12c4 service neutron] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:11:42.682330 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 499/997] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:11:42 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:11:45.731877 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e1206a-60e0-4062-9af2-df0c017d73fd None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:11:45.732424 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e1206a-60e0-4062-9af2-df0c017d73fd None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:11:45.732424 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e1206a-60e0-4062-9af2-df0c017d73fd None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:11:45.733763 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 499/998] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:11:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:11:45.739081 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f067690f-1474-4d67-a54a-df8162161a06 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:11:45.739582 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f067690f-1474-4d67-a54a-df8162161a06 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:11:45.739687 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f067690f-1474-4d67-a54a-df8162161a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:11:45.807967 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-f067690f-1474-4d67-a54a-df8162161a06 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:11:45.809142 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-f067690f-1474-4d67-a54a-df8162161a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:11:45.875474 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:11:45.917675 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 500/999] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:11:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:11:46.210054 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c781e217-6812-4041-873b-5295bee23d9c None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:11:46.210431 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c781e217-6812-4041-873b-5295bee23d9c None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:11:46.210760 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c781e217-6812-4041-873b-5295bee23d9c None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:11:46.212158 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 500/1000] 10.222.0.36 () {54 vars in 964 bytes} [Wed Oct 12 00:11:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 00:11:46.218041 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047497f3-1ef0-4747-9225-129ef1256ccc None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:11:46.218397 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047497f3-1ef0-4747-9225-129ef1256ccc None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:11:46.218776 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047497f3-1ef0-4747-9225-129ef1256ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:11:46.294129 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-047497f3-1ef0-4747-9225-129ef1256ccc None None] MFA Rules not processed for user `51215dab618141b4991e232724094140`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:11:46.295646 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-047497f3-1ef0-4747-9225-129ef1256ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:11:46.371989 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 501/1001] 10.222.0.36 () {58 vars in 1061 bytes} [Wed Oct 12 00:11:46 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:11:46.381390 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c781e217-6812-4041-873b-5295bee23d9c None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:11:46.383022 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-c781e217-6812-4041-873b-5295bee23d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:11:46.403281 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:11:46.491502 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c781e217-6812-4041-873b-5295bee23d9c None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:11:46.567267 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0bfd91-a061-4b4e-bfe7-38a888f2af20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:11:46.567900 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0bfd91-a061-4b4e-bfe7-38a888f2af20 service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:11:46.568177 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0bfd91-a061-4b4e-bfe7-38a888f2af20 service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:11:46.568451 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0bfd91-a061-4b4e-bfe7-38a888f2af20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:11:46.570041 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-8f0bfd91-a061-4b4e-bfe7-38a888f2af20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:11:46.640319 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:11:46.676651 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0bfd91-a061-4b4e-bfe7-38a888f2af20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:11:46.678089 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0bfd91-a061-4b4e-bfe7-38a888f2af20 service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:11:46.696264 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 501/1002] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:11:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:11:49.091942 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ae0d44-4b07-4320-ab99-a29d535f3b2e None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:11:49.092496 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ae0d44-4b07-4320-ab99-a29d535f3b2e None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:11:49.092496 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ae0d44-4b07-4320-ab99-a29d535f3b2e None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:11:49.094310 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 502/1003] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:11:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:11:49.099496 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6021fea-cdd5-4d98-bcbb-623bd121ce6f None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:11:49.099935 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6021fea-cdd5-4d98-bcbb-623bd121ce6f None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:11:49.100186 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6021fea-cdd5-4d98-bcbb-623bd121ce6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:11:49.165515 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-e6021fea-cdd5-4d98-bcbb-623bd121ce6f None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:11:49.166631 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-e6021fea-cdd5-4d98-bcbb-623bd121ce6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:11:49.264257 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 502/1004] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:11:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:11:49.545090 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ae0d44-4b07-4320-ab99-a29d535f3b2e None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:11:49.546692 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-64ae0d44-4b07-4320-ab99-a29d535f3b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:11:49.608304 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:11:49.651926 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ae0d44-4b07-4320-ab99-a29d535f3b2e None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:11:49.726195 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3782aba6-a38b-4635-a870-65024382c0d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:11:49.726801 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3782aba6-a38b-4635-a870-65024382c0d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:11:49.727082 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3782aba6-a38b-4635-a870-65024382c0d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:11:49.727352 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3782aba6-a38b-4635-a870-65024382c0d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:11:49.728982 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-3782aba6-a38b-4635-a870-65024382c0d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:11:49.836276 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:11:49.838579 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3782aba6-a38b-4635-a870-65024382c0d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:11:49.840058 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3782aba6-a38b-4635-a870-65024382c0d3 service neutron] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:11:49.857540 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 503/1005] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:11:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:11:52.059052 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9241fcb-731e-4735-9016-a656b577fc20 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:11:52.059644 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9241fcb-731e-4735-9016-a656b577fc20 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:11:52.059644 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9241fcb-731e-4735-9016-a656b577fc20 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:11:52.060980 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 503/1006] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:11:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:11:52.066896 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f98013-ba41-4458-95b6-bb83d0161e4f None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:11:52.067340 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f98013-ba41-4458-95b6-bb83d0161e4f None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:11:52.067457 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f98013-ba41-4458-95b6-bb83d0161e4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:11:52.134683 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-e6f98013-ba41-4458-95b6-bb83d0161e4f None None] MFA Rules not processed for user `216d4237a0a44e7eaf170e6dcd700023`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:11:52.135929 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-e6f98013-ba41-4458-95b6-bb83d0161e4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:11:52.236590 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 504/1007] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:11:52 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:11:52.522904 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9241fcb-731e-4735-9016-a656b577fc20 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:11:52.524576 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-c9241fcb-731e-4735-9016-a656b577fc20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:11:52.556022 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:11:52.626704 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9241fcb-731e-4735-9016-a656b577fc20 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:11:52.700621 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4ef749-14c1-4abd-81d5-3e8195729fc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:11:52.701226 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ef749-14c1-4abd-81d5-3e8195729fc8 service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:11:52.701501 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ef749-14c1-4abd-81d5-3e8195729fc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:11:52.701769 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ef749-14c1-4abd-81d5-3e8195729fc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:11:52.703415 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-8e4ef749-14c1-4abd-81d5-3e8195729fc8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:11:52.779297 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:11:52.813306 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4ef749-14c1-4abd-81d5-3e8195729fc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:11:52.814759 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4ef749-14c1-4abd-81d5-3e8195729fc8 service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:11:52.836126 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 504/1008] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:11:52 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:11:55.368119 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff12880-c4b5-4195-98f6-4226b819c9fd None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:11:55.368682 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff12880-c4b5-4195-98f6-4226b819c9fd None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:11:55.368682 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff12880-c4b5-4195-98f6-4226b819c9fd None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:11:55.370002 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 505/1009] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:11:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:11:55.375489 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c43818-33d1-4bc2-ad1c-1705537b8a6d None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:11:55.375907 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c43818-33d1-4bc2-ad1c-1705537b8a6d None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:11:55.376016 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c43818-33d1-4bc2-ad1c-1705537b8a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:11:55.441194 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-c9c43818-33d1-4bc2-ad1c-1705537b8a6d None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:11:55.442400 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-c9c43818-33d1-4bc2-ad1c-1705537b8a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:11:55.538199 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 505/1010] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:11:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:11:55.822774 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff12880-c4b5-4195-98f6-4226b819c9fd None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:11:55.824354 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-fff12880-c4b5-4195-98f6-4226b819c9fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:11:55.893811 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:11:55.927002 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff12880-c4b5-4195-98f6-4226b819c9fd None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:11:55.999192 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cdd194-f583-43fa-8fc6-7805aa57fc76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:11:56.000309 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cdd194-f583-43fa-8fc6-7805aa57fc76 service neutron] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:11:56.000583 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cdd194-f583-43fa-8fc6-7805aa57fc76 service neutron] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:11:56.000848 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cdd194-f583-43fa-8fc6-7805aa57fc76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:11:56.002417 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-b1cdd194-f583-43fa-8fc6-7805aa57fc76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:11:56.108549 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cdd194-f583-43fa-8fc6-7805aa57fc76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:11:56.110488 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cdd194-f583-43fa-8fc6-7805aa57fc76 service neutron] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:11:56.120216 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:11:56.128886 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 506/1011] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:11:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:11:58.303741 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33df06f9-7977-43b1-9238-e25a01045e7f None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:11:58.304446 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33df06f9-7977-43b1-9238-e25a01045e7f None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:11:58.304446 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33df06f9-7977-43b1-9238-e25a01045e7f None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:11:58.305669 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 506/1012] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:11:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:11:58.311241 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294a5702-b405-41b4-94bf-3f29ef94c3c6 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:11:58.311665 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294a5702-b405-41b4-94bf-3f29ef94c3c6 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:11:58.312437 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294a5702-b405-41b4-94bf-3f29ef94c3c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:11:58.377572 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-294a5702-b405-41b4-94bf-3f29ef94c3c6 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:11:58.378764 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-294a5702-b405-41b4-94bf-3f29ef94c3c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:11:58.474128 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 507/1013] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:11:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:11:58.756118 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33df06f9-7977-43b1-9238-e25a01045e7f None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:11:58.757706 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-33df06f9-7977-43b1-9238-e25a01045e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:11:58.799658 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:11:58.861191 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33df06f9-7977-43b1-9238-e25a01045e7f None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:11:58.931330 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c18e2a-09e0-4d98-bae7-c9fb45426975 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:11:58.931949 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c18e2a-09e0-4d98-bae7-c9fb45426975 service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:11:58.932223 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c18e2a-09e0-4d98-bae7-c9fb45426975 service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:11:58.932490 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c18e2a-09e0-4d98-bae7-c9fb45426975 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:11:58.934063 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-a6c18e2a-09e0-4d98-bae7-c9fb45426975 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:11:59.013853 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:11:59.039863 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c18e2a-09e0-4d98-bae7-c9fb45426975 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:11:59.041325 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c18e2a-09e0-4d98-bae7-c9fb45426975 service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:11:59.058841 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 507/1014] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:11:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:12:01.755658 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9677a063-ebcb-47c4-99f2-412e002c66ea None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:12:01.756143 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9677a063-ebcb-47c4-99f2-412e002c66ea None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:12:01.759524 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9677a063-ebcb-47c4-99f2-412e002c66ea None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:12:01.759524 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 508/1015] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:12:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:12:01.762740 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b65456-d91c-4670-80b4-f9a2a4c61a04 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:12:01.763135 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b65456-d91c-4670-80b4-f9a2a4c61a04 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:12:01.763293 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b65456-d91c-4670-80b4-f9a2a4c61a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:12:01.829407 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-46b65456-d91c-4670-80b4-f9a2a4c61a04 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:12:01.830622 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-46b65456-d91c-4670-80b4-f9a2a4c61a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:12:01.928462 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 508/1016] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:12:01 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:12:02.209655 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9677a063-ebcb-47c4-99f2-412e002c66ea None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:12:02.211182 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-9677a063-ebcb-47c4-99f2-412e002c66ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:12:02.286084 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:12:02.313053 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9677a063-ebcb-47c4-99f2-412e002c66ea None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:12:02.386863 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f79bea8-6327-4e60-b555-4ac1f773df0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:12:02.387593 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f79bea8-6327-4e60-b555-4ac1f773df0e service neutron] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:12:02.387870 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f79bea8-6327-4e60-b555-4ac1f773df0e service neutron] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:12:02.388138 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f79bea8-6327-4e60-b555-4ac1f773df0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:12:02.389982 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-2f79bea8-6327-4e60-b555-4ac1f773df0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:12:02.496079 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f79bea8-6327-4e60-b555-4ac1f773df0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:12:02.497441 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f79bea8-6327-4e60-b555-4ac1f773df0e service neutron] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:12:02.516571 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 509/1017] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:12:02 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:12:06.706194 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23f7a09-91d8-41cc-bbc4-480c12613e6f None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:12:06.706744 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23f7a09-91d8-41cc-bbc4-480c12613e6f None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:12:06.706744 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23f7a09-91d8-41cc-bbc4-480c12613e6f None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:12:06.708262 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 509/1018] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:12:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:12:06.713401 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd290fe-d100-4ea8-8b87-fa4a0647731e None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:12:06.713827 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd290fe-d100-4ea8-8b87-fa4a0647731e None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:12:06.713948 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd290fe-d100-4ea8-8b87-fa4a0647731e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:12:06.720148 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:12:06.782349 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-4cd290fe-d100-4ea8-8b87-fa4a0647731e None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:12:06.783939 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-4cd290fe-d100-4ea8-8b87-fa4a0647731e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:12:06.881267 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 510/1019] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:12:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 00:12:07.159846 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23f7a09-91d8-41cc-bbc4-480c12613e6f None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:12:07.161409 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-d23f7a09-91d8-41cc-bbc4-480c12613e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:12:07.210002 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:12:07.263973 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23f7a09-91d8-41cc-bbc4-480c12613e6f None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:12:07.338028 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f64a6a-adb8-4bae-bf3d-3d4248f69a3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:12:07.338626 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f64a6a-adb8-4bae-bf3d-3d4248f69a3f service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:12:07.338898 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f64a6a-adb8-4bae-bf3d-3d4248f69a3f service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:12:07.339164 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f64a6a-adb8-4bae-bf3d-3d4248f69a3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:12:07.340769 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-73f64a6a-adb8-4bae-bf3d-3d4248f69a3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:12:07.433719 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:12:07.450125 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f64a6a-adb8-4bae-bf3d-3d4248f69a3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:12:07.455398 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f64a6a-adb8-4bae-bf3d-3d4248f69a3f service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:12:07.470532 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 510/1020] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:12:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:12:10.916603 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5e7252-f7bb-4bcf-b276-2cd100c8d290 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:12:10.917143 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5e7252-f7bb-4bcf-b276-2cd100c8d290 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:12:10.917143 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5e7252-f7bb-4bcf-b276-2cd100c8d290 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:12:10.918487 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 511/1021] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:12:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:12:10.923948 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed419a09-e425-4687-9e69-2a82a90e3b41 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:12:10.924373 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed419a09-e425-4687-9e69-2a82a90e3b41 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:12:10.924484 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed419a09-e425-4687-9e69-2a82a90e3b41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:12:10.989925 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-ed419a09-e425-4687-9e69-2a82a90e3b41 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:12:10.991420 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-ed419a09-e425-4687-9e69-2a82a90e3b41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:12:11.087779 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 511/1022] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:12:10 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:12:11.370085 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5e7252-f7bb-4bcf-b276-2cd100c8d290 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:12:11.371841 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-fe5e7252-f7bb-4bcf-b276-2cd100c8d290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:12:11.464697 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:12:11.481860 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5e7252-f7bb-4bcf-b276-2cd100c8d290 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:12:11.560808 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d56f97-e7f7-418b-ad0b-6263dd3d6392 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:12:11.561417 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d56f97-e7f7-418b-ad0b-6263dd3d6392 service neutron] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:12:11.561699 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d56f97-e7f7-418b-ad0b-6263dd3d6392 service neutron] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:12:11.562009 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d56f97-e7f7-418b-ad0b-6263dd3d6392 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:12:11.563636 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-88d56f97-e7f7-418b-ad0b-6263dd3d6392 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:12:11.678848 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d56f97-e7f7-418b-ad0b-6263dd3d6392 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:12:11.680334 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d56f97-e7f7-418b-ad0b-6263dd3d6392 service neutron] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:12:11.697941 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 512/1023] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:12:11 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:12:16.080009 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbaca80-3259-49f8-b470-555ea386f8f0 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:12:16.080465 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbaca80-3259-49f8-b470-555ea386f8f0 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:12:16.080565 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbaca80-3259-49f8-b470-555ea386f8f0 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:12:16.081878 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 512/1024] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:12:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:12:16.087580 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4c0b87-bd99-4d92-b8b5-71dc6bc48e11 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:12:16.088052 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4c0b87-bd99-4d92-b8b5-71dc6bc48e11 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:12:16.088145 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4c0b87-bd99-4d92-b8b5-71dc6bc48e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:12:16.104018 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:12:16.156416 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-8c4c0b87-bd99-4d92-b8b5-71dc6bc48e11 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:12:16.157606 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-8c4c0b87-bd99-4d92-b8b5-71dc6bc48e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:12:16.256789 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 513/1025] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:12:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:12:16.538993 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbaca80-3259-49f8-b470-555ea386f8f0 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:12:16.540577 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-1bbaca80-3259-49f8-b470-555ea386f8f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:12:16.597790 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:12:16.642941 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbaca80-3259-49f8-b470-555ea386f8f0 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:12:16.716423 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7575a0c-b614-4f0b-968f-4a5adf08351e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:12:16.717026 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7575a0c-b614-4f0b-968f-4a5adf08351e service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:12:16.717296 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7575a0c-b614-4f0b-968f-4a5adf08351e service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:12:16.717561 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7575a0c-b614-4f0b-968f-4a5adf08351e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:12:16.719172 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-c7575a0c-b614-4f0b-968f-4a5adf08351e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:12:16.830305 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:12:16.833895 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7575a0c-b614-4f0b-968f-4a5adf08351e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:12:16.835336 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7575a0c-b614-4f0b-968f-4a5adf08351e service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:12:16.853499 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 513/1026] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:12:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:12:21.750967 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60be3fe6-98dc-4ac8-a53b-2075895a92ec None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:12:21.751478 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60be3fe6-98dc-4ac8-a53b-2075895a92ec None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:12:21.751765 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60be3fe6-98dc-4ac8-a53b-2075895a92ec None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:12:21.753473 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 514/1027] 10.222.0.36 () {54 vars in 921 bytes} [Wed Oct 12 00:12:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:12:21.758748 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a805796e-eb4c-4e1f-8757-8b1a3d258f4e None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:12:21.759589 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a805796e-eb4c-4e1f-8757-8b1a3d258f4e None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:12:21.759923 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a805796e-eb4c-4e1f-8757-8b1a3d258f4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:12:21.936024 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-a805796e-eb4c-4e1f-8757-8b1a3d258f4e None None] MFA Rules not processed for user `1e528da752ed4c78bc747215909f6569`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:12:21.937873 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-a805796e-eb4c-4e1f-8757-8b1a3d258f4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:12:22.137578 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 514/1028] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Oct 12 00:12:21 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 380 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:12:26.255198 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=59 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:12:26.325632 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc8770a-e846-43a3-b0ef-d6258ec481c1 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:12:26.325919 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc8770a-e846-43a3-b0ef-d6258ec481c1 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:12:26.326180 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc8770a-e846-43a3-b0ef-d6258ec481c1 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:12:26.328197 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 515/1029] 10.222.0.36 () {54 vars in 921 bytes} [Wed Oct 12 00:12:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:12:26.333021 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3329a0a-4845-4760-aba2-9d201fd6809a None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:12:26.333805 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3329a0a-4845-4760-aba2-9d201fd6809a None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:12:26.334264 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3329a0a-4845-4760-aba2-9d201fd6809a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:12:26.394777 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:12:26.422407 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-f3329a0a-4845-4760-aba2-9d201fd6809a None None] MFA Rules not processed for user `1e528da752ed4c78bc747215909f6569`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:12:26.423889 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-f3329a0a-4845-4760-aba2-9d201fd6809a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:12:26.680657 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 515/1030] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Oct 12 00:12:26 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:12:27.722708 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a62843-3d21-470f-9027-bff9796ba8e4 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:12:27.723261 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a62843-3d21-470f-9027-bff9796ba8e4 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:12:27.723261 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a62843-3d21-470f-9027-bff9796ba8e4 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:12:27.724636 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 516/1031] 10.222.0.36 () {54 vars in 921 bytes} [Wed Oct 12 00:12:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:12:27.729617 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6ee674-ccc6-45fc-84b1-5fd7d1ff4643 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:12:27.730010 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6ee674-ccc6-45fc-84b1-5fd7d1ff4643 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:12:27.730191 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6ee674-ccc6-45fc-84b1-5fd7d1ff4643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:12:27.808247 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-cb6ee674-ccc6-45fc-84b1-5fd7d1ff4643 None None] MFA Rules not processed for user `1e528da752ed4c78bc747215909f6569`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:12:27.809499 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-cb6ee674-ccc6-45fc-84b1-5fd7d1ff4643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:12:27.851174 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:12:27.919505 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 516/1032] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Oct 12 00:12:27 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 00:12:31.564902 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b2dc62-0951-4f38-af5e-877737127023 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:12:31.565424 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b2dc62-0951-4f38-af5e-877737127023 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:12:31.565424 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b2dc62-0951-4f38-af5e-877737127023 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:12:31.566886 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 517/1033] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:12:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 00:12:31.574242 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c34a1d-422f-4b70-b606-1ccdfde9e9d8 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:12:31.574702 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c34a1d-422f-4b70-b606-1ccdfde9e9d8 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:12:31.574794 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c34a1d-422f-4b70-b606-1ccdfde9e9d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:12:31.642542 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-61c34a1d-422f-4b70-b606-1ccdfde9e9d8 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:12:31.643689 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-61c34a1d-422f-4b70-b606-1ccdfde9e9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:12:31.753342 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 517/1034] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:12:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:12:32.056271 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c161110-4a19-4845-bae0-028f92626489 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:12:32.056958 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c161110-4a19-4845-bae0-028f92626489 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:12:32.057489 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c161110-4a19-4845-bae0-028f92626489 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:12:32.059749 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 518/1035] 10.222.0.36 () {54 vars in 961 bytes} [Wed Oct 12 00:12:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:12:32.067528 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0170dd68-8e43-45a9-8a1a-7edf8c3fa96a None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:12:32.073280 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0170dd68-8e43-45a9-8a1a-7edf8c3fa96a None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:12:32.073280 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0170dd68-8e43-45a9-8a1a-7edf8c3fa96a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:12:32.086929 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:12:32.148788 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-0170dd68-8e43-45a9-8a1a-7edf8c3fa96a None None] MFA Rules not processed for user `a515b75aa3944912ad2364ffd7924f40`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:12:32.150274 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-0170dd68-8e43-45a9-8a1a-7edf8c3fa96a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:12:32.289790 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 518/1036] 10.222.0.36 () {58 vars in 1058 bytes} [Wed Oct 12 00:12:32 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 00:12:32.298315 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c161110-4a19-4845-bae0-028f92626489 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:12:32.300179 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-6c161110-4a19-4845-bae0-028f92626489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:12:32.456531 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c161110-4a19-4845-bae0-028f92626489 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:12:32.545442 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:12:32.586925 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623f0db7-14cc-4e0c-a35e-0f8c3f1a114b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a515b75aa3944912ad2364ffd7924f40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:12:32.587601 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623f0db7-14cc-4e0c-a35e-0f8c3f1a114b service nova] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:12:32.587900 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623f0db7-14cc-4e0c-a35e-0f8c3f1a114b service nova] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:12:32.588081 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623f0db7-14cc-4e0c-a35e-0f8c3f1a114b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:12:32.589840 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-623f0db7-14cc-4e0c-a35e-0f8c3f1a114b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:12:32.691648 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623f0db7-14cc-4e0c-a35e-0f8c3f1a114b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:12:32.693298 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623f0db7-14cc-4e0c-a35e-0f8c3f1a114b service nova] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:12:32.713293 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 519/1037] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:12:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 00:12:35.090714 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d3abb0-6703-4455-83a8-03373a3cc419 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:12:35.091233 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d3abb0-6703-4455-83a8-03373a3cc419 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:12:35.091233 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d3abb0-6703-4455-83a8-03373a3cc419 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:12:35.093073 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 519/1038] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:12:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:12:35.100837 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f10713c-e241-46ff-b48d-17625565f9c0 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:12:35.100837 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f10713c-e241-46ff-b48d-17625565f9c0 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:12:35.100837 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f10713c-e241-46ff-b48d-17625565f9c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:12:35.179235 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-4f10713c-e241-46ff-b48d-17625565f9c0 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:12:35.180507 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-4f10713c-e241-46ff-b48d-17625565f9c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:12:35.221851 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:12:35.285978 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 520/1039] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:12:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:12:35.579142 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d3abb0-6703-4455-83a8-03373a3cc419 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:12:35.580698 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-c0d3abb0-6703-4455-83a8-03373a3cc419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:12:35.649875 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:12:35.753507 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d3abb0-6703-4455-83a8-03373a3cc419 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:12:35.880191 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a135aad-d35e-4064-aef2-fe1108b456f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a515b75aa3944912ad2364ffd7924f40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:12:35.880807 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a135aad-d35e-4064-aef2-fe1108b456f6 service nova] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:12:35.881080 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a135aad-d35e-4064-aef2-fe1108b456f6 service nova] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:12:35.881344 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a135aad-d35e-4064-aef2-fe1108b456f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:12:35.883092 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-6a135aad-d35e-4064-aef2-fe1108b456f6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:12:35.906341 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:12:35.976656 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a135aad-d35e-4064-aef2-fe1108b456f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:12:35.978268 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a135aad-d35e-4064-aef2-fe1108b456f6 service nova] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:12:35.996829 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 520/1040] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:12:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:12:38.595098 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf93975-7379-48f3-b922-347ab5e69aa0 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:12:38.595681 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf93975-7379-48f3-b922-347ab5e69aa0 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:12:38.595681 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf93975-7379-48f3-b922-347ab5e69aa0 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:12:38.597194 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 521/1041] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:12:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:12:38.602200 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edad355-0d70-4b31-82fb-3b708480edfa None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:12:38.602617 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edad355-0d70-4b31-82fb-3b708480edfa None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:12:38.602793 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edad355-0d70-4b31-82fb-3b708480edfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:12:38.672054 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-5edad355-0d70-4b31-82fb-3b708480edfa None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:12:38.673296 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-5edad355-0d70-4b31-82fb-3b708480edfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:12:38.775326 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:12:38.778083 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 521/1042] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:12:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:12:39.066953 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39041af-85bb-406f-9b8c-6a818b276b71 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:12:39.067513 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39041af-85bb-406f-9b8c-6a818b276b71 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:12:39.067850 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39041af-85bb-406f-9b8c-6a818b276b71 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:12:39.069488 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 522/1043] 10.222.0.36 () {54 vars in 961 bytes} [Wed Oct 12 00:12:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:12:39.074895 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd344258-7107-43cd-a2e1-0c3018c73003 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:12:39.079628 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd344258-7107-43cd-a2e1-0c3018c73003 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:12:39.079628 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd344258-7107-43cd-a2e1-0c3018c73003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:12:39.157540 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-bd344258-7107-43cd-a2e1-0c3018c73003 None None] MFA Rules not processed for user `a515b75aa3944912ad2364ffd7924f40`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:12:39.158892 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-bd344258-7107-43cd-a2e1-0c3018c73003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:12:39.287742 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 522/1044] 10.222.0.36 () {58 vars in 1058 bytes} [Wed Oct 12 00:12:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:12:39.297275 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39041af-85bb-406f-9b8c-6a818b276b71 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:12:39.298940 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-f39041af-85bb-406f-9b8c-6a818b276b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:12:39.458013 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39041af-85bb-406f-9b8c-6a818b276b71 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:12:39.479488 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:12:39.578568 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ee6df8-2f55-464e-b01b-b25eb93ebd56 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a515b75aa3944912ad2364ffd7924f40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:12:39.579199 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ee6df8-2f55-464e-b01b-b25eb93ebd56 service nova] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:12:39.579503 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ee6df8-2f55-464e-b01b-b25eb93ebd56 service nova] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:12:39.579772 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ee6df8-2f55-464e-b01b-b25eb93ebd56 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:12:39.581490 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-17ee6df8-2f55-464e-b01b-b25eb93ebd56 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:12:39.677321 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ee6df8-2f55-464e-b01b-b25eb93ebd56 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:12:39.678908 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ee6df8-2f55-464e-b01b-b25eb93ebd56 service nova] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:12:39.689399 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c6941a-7bc6-4f04-b539-daa4a34229d8 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:12:39.689898 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c6941a-7bc6-4f04-b539-daa4a34229d8 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:12:39.689994 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c6941a-7bc6-4f04-b539-daa4a34229d8 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:12:39.691448 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 523/1045] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:12:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:12:39.696193 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c491899-95d4-41ff-97d3-52e539be9d07 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:12:39.696568 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c491899-95d4-41ff-97d3-52e539be9d07 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:12:39.696841 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c491899-95d4-41ff-97d3-52e539be9d07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:12:39.698716 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 523/1046] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:12:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:12:39.757826 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:12:39.765760 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-3c491899-95d4-41ff-97d3-52e539be9d07 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:12:39.766972 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-3c491899-95d4-41ff-97d3-52e539be9d07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:12:39.882252 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 524/1047] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:12:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 00:12:40.222616 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb41671-dc6c-44f2-80f1-5ca0c9ef275c None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:12:40.223017 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb41671-dc6c-44f2-80f1-5ca0c9ef275c None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:12:40.223415 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb41671-dc6c-44f2-80f1-5ca0c9ef275c None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:12:40.224993 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 524/1048] 10.222.0.36 () {54 vars in 963 bytes} [Wed Oct 12 00:12:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:12:40.229821 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29e6783-a0a0-4a4e-b808-8a8f3d00efe1 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:12:40.230244 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29e6783-a0a0-4a4e-b808-8a8f3d00efe1 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:12:40.230640 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29e6783-a0a0-4a4e-b808-8a8f3d00efe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:12:40.312012 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-b29e6783-a0a0-4a4e-b808-8a8f3d00efe1 None None] MFA Rules not processed for user `7f87febbbaa94626807a98da4a771ab1`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:12:40.313234 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-b29e6783-a0a0-4a4e-b808-8a8f3d00efe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:12:40.385390 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:12:40.393116 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 525/1049] 10.222.0.36 () {58 vars in 1060 bytes} [Wed Oct 12 00:12:40 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:12:40.400677 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb41671-dc6c-44f2-80f1-5ca0c9ef275c None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:12:40.406372 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-4bb41671-dc6c-44f2-80f1-5ca0c9ef275c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:12:40.434424 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:12:40.512992 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb41671-dc6c-44f2-80f1-5ca0c9ef275c None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:12:40.594054 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e58ccdc-3dbb-49f6-b71d-b0738c4db5cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:12:40.594860 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e58ccdc-3dbb-49f6-b71d-b0738c4db5cf service cinder] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:12:40.595137 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e58ccdc-3dbb-49f6-b71d-b0738c4db5cf service cinder] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:12:40.595501 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e58ccdc-3dbb-49f6-b71d-b0738c4db5cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:12:40.597189 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-3e58ccdc-3dbb-49f6-b71d-b0738c4db5cf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:12:40.678336 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:12:40.711934 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e58ccdc-3dbb-49f6-b71d-b0738c4db5cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:12:40.713365 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e58ccdc-3dbb-49f6-b71d-b0738c4db5cf service cinder] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:12:40.743699 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 525/1050] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:12:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:12:42.065940 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb2f52a-8038-4054-b935-cdd5380e22e4 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:12:42.066526 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb2f52a-8038-4054-b935-cdd5380e22e4 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:12:42.066526 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb2f52a-8038-4054-b935-cdd5380e22e4 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:12:42.067845 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 526/1051] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:12:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:12:42.073012 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b57f81-9adf-446b-92d4-b6a804a9d3dc None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:12:42.073400 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b57f81-9adf-446b-92d4-b6a804a9d3dc None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:12:42.073553 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b57f81-9adf-446b-92d4-b6a804a9d3dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:12:42.141508 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-f2b57f81-9adf-446b-92d4-b6a804a9d3dc None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:12:42.142716 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-f2b57f81-9adf-446b-92d4-b6a804a9d3dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:12:42.238469 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 526/1052] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:12:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:12:42.529350 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb2f52a-8038-4054-b935-cdd5380e22e4 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:12:42.531397 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-5eb2f52a-8038-4054-b935-cdd5380e22e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:12:42.686003 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb2f52a-8038-4054-b935-cdd5380e22e4 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:12:42.755853 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:12:42.807416 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b802007a-2f27-4199-9377-bc82b16bbdad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a515b75aa3944912ad2364ffd7924f40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:12:42.807416 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b802007a-2f27-4199-9377-bc82b16bbdad service nova] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:12:42.807658 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b802007a-2f27-4199-9377-bc82b16bbdad service nova] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:12:42.808043 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b802007a-2f27-4199-9377-bc82b16bbdad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:12:42.809649 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-b802007a-2f27-4199-9377-bc82b16bbdad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:12:42.903865 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b802007a-2f27-4199-9377-bc82b16bbdad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:12:42.906053 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b802007a-2f27-4199-9377-bc82b16bbdad service nova] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:12:42.925622 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 527/1053] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:12:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 00:12:45.228001 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d60a8ce-4088-4cc7-8fd0-ed3bb946f197 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:12:45.228869 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d60a8ce-4088-4cc7-8fd0-ed3bb946f197 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:12:45.229325 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d60a8ce-4088-4cc7-8fd0-ed3bb946f197 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:12:45.230820 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 527/1054] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:12:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:12:45.236361 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc11cf-e05a-4bdc-b522-a82b6b5b386a None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:12:45.236771 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc11cf-e05a-4bdc-b522-a82b6b5b386a None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:12:45.236914 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc11cf-e05a-4bdc-b522-a82b6b5b386a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:12:45.304935 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-befc11cf-e05a-4bdc-b522-a82b6b5b386a None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:12:45.306190 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-befc11cf-e05a-4bdc-b522-a82b6b5b386a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:12:45.335916 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:12:45.405759 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 528/1055] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:12:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:12:45.689984 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d60a8ce-4088-4cc7-8fd0-ed3bb946f197 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:12:45.691506 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-8d60a8ce-4088-4cc7-8fd0-ed3bb946f197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:12:45.731009 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:12:45.851215 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d60a8ce-4088-4cc7-8fd0-ed3bb946f197 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:12:45.969775 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:12:45.975303 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21e25a9-889b-4584-af0d-96a088dcc46a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a515b75aa3944912ad2364ffd7924f40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:12:45.976047 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e25a9-889b-4584-af0d-96a088dcc46a service nova] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:12:45.976369 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e25a9-889b-4584-af0d-96a088dcc46a service nova] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:12:45.976700 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e25a9-889b-4584-af0d-96a088dcc46a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:12:45.978529 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-e21e25a9-889b-4584-af0d-96a088dcc46a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:12:46.080117 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21e25a9-889b-4584-af0d-96a088dcc46a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:12:46.081780 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21e25a9-889b-4584-af0d-96a088dcc46a service nova] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:12:46.101624 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 528/1056] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:12:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 00:12:48.502581 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faf20df-77b9-432a-bde6-0c716daeef59 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:12:48.503463 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faf20df-77b9-432a-bde6-0c716daeef59 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:12:48.503751 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faf20df-77b9-432a-bde6-0c716daeef59 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:12:48.505142 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 529/1057] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:12:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:12:48.510490 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7762d64d-c1e7-4bad-bf51-61fd5d6ca77a None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:12:48.510951 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7762d64d-c1e7-4bad-bf51-61fd5d6ca77a None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:12:48.511048 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7762d64d-c1e7-4bad-bf51-61fd5d6ca77a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:12:48.581137 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-7762d64d-c1e7-4bad-bf51-61fd5d6ca77a None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:12:48.582382 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-7762d64d-c1e7-4bad-bf51-61fd5d6ca77a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:12:48.649010 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:12:48.688117 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 529/1058] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:12:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:12:48.987205 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faf20df-77b9-432a-bde6-0c716daeef59 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:12:48.988816 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-3faf20df-77b9-432a-bde6-0c716daeef59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:12:49.136690 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faf20df-77b9-432a-bde6-0c716daeef59 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:12:49.150022 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:12:49.258320 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc9c80f-0ebb-42c1-9102-d68d5f605bb7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a515b75aa3944912ad2364ffd7924f40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:12:49.258853 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc9c80f-0ebb-42c1-9102-d68d5f605bb7 service nova] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:12:49.259192 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc9c80f-0ebb-42c1-9102-d68d5f605bb7 service nova] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:12:49.259425 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc9c80f-0ebb-42c1-9102-d68d5f605bb7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:12:49.261028 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-6dc9c80f-0ebb-42c1-9102-d68d5f605bb7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:12:49.352329 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc9c80f-0ebb-42c1-9102-d68d5f605bb7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:12:49.353940 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc9c80f-0ebb-42c1-9102-d68d5f605bb7 service nova] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:12:49.372846 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 530/1059] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:12:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:12:51.724307 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddf0cff-fb0e-48ea-9757-1c06a2400d7a None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:12:51.724820 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddf0cff-fb0e-48ea-9757-1c06a2400d7a None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:12:51.725122 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddf0cff-fb0e-48ea-9757-1c06a2400d7a None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:12:51.726861 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 530/1060] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:12:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:12:51.732269 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de88df45-cbca-4b17-926b-87ba2bd19f73 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:12:51.732787 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de88df45-cbca-4b17-926b-87ba2bd19f73 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:12:51.732893 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de88df45-cbca-4b17-926b-87ba2bd19f73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:12:51.754845 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:12:51.801011 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-de88df45-cbca-4b17-926b-87ba2bd19f73 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:12:51.802263 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-de88df45-cbca-4b17-926b-87ba2bd19f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:12:51.910363 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 531/1061] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:12:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:12:52.204871 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddf0cff-fb0e-48ea-9757-1c06a2400d7a None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:12:52.206527 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-7ddf0cff-fb0e-48ea-9757-1c06a2400d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:12:52.368863 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddf0cff-fb0e-48ea-9757-1c06a2400d7a None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:12:52.428442 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:12:52.489245 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398e01f4-d4fc-44ca-891f-6a7d44d9c827 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a515b75aa3944912ad2364ffd7924f40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:12:52.489770 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e01f4-d4fc-44ca-891f-6a7d44d9c827 service nova] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:12:52.490043 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e01f4-d4fc-44ca-891f-6a7d44d9c827 service nova] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:12:52.490308 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e01f4-d4fc-44ca-891f-6a7d44d9c827 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:12:52.491921 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-398e01f4-d4fc-44ca-891f-6a7d44d9c827 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:12:52.597868 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398e01f4-d4fc-44ca-891f-6a7d44d9c827 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:12:52.600007 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398e01f4-d4fc-44ca-891f-6a7d44d9c827 service nova] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:12:52.621129 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 531/1062] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:12:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:12:54.856043 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f504be-55b5-4083-8252-a7284b46ad16 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:12:54.856571 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f504be-55b5-4083-8252-a7284b46ad16 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:12:54.856663 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f504be-55b5-4083-8252-a7284b46ad16 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:12:54.858467 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 532/1063] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:12:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:12:54.863874 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9044f25c-ba8e-49e1-9f24-6784fea579da None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:12:54.864310 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9044f25c-ba8e-49e1-9f24-6784fea579da None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:12:54.864421 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9044f25c-ba8e-49e1-9f24-6784fea579da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:12:54.936053 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-9044f25c-ba8e-49e1-9f24-6784fea579da None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:12:54.937267 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-9044f25c-ba8e-49e1-9f24-6784fea579da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:12:54.945807 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:12:55.034234 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 532/1064] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:12:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:12:55.318652 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f504be-55b5-4083-8252-a7284b46ad16 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:12:55.320184 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-20f504be-55b5-4083-8252-a7284b46ad16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:12:55.411969 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:12:55.466422 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f504be-55b5-4083-8252-a7284b46ad16 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:12:55.578596 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be28deb-a737-4c58-93e0-fff6eee1acfb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a515b75aa3944912ad2364ffd7924f40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:12:55.579209 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be28deb-a737-4c58-93e0-fff6eee1acfb service nova] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:12:55.579515 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be28deb-a737-4c58-93e0-fff6eee1acfb service nova] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:12:55.579787 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be28deb-a737-4c58-93e0-fff6eee1acfb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:12:55.581378 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-8be28deb-a737-4c58-93e0-fff6eee1acfb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:12:55.640391 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:12:55.671474 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be28deb-a737-4c58-93e0-fff6eee1acfb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:12:55.673061 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be28deb-a737-4c58-93e0-fff6eee1acfb service nova] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:12:55.690760 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 533/1065] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:12:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:12:56.164270 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f55710e-fa68-4f30-b196-bd8801253c3c None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:12:56.164822 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f55710e-fa68-4f30-b196-bd8801253c3c None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:12:56.164822 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f55710e-fa68-4f30-b196-bd8801253c3c None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:12:56.166045 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 533/1066] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:12:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:12:56.171824 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5875193e-d55e-499f-b400-cc5ed1ef80d9 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:12:56.172240 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5875193e-d55e-499f-b400-cc5ed1ef80d9 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:12:56.172371 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5875193e-d55e-499f-b400-cc5ed1ef80d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:12:56.238071 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-5875193e-d55e-499f-b400-cc5ed1ef80d9 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:12:56.239269 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-5875193e-d55e-499f-b400-cc5ed1ef80d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:12:56.339656 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 534/1067] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:12:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:12:56.640388 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c80c383-591a-47e0-ad10-993832ccf1be None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:12:56.640742 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c80c383-591a-47e0-ad10-993832ccf1be None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:12:56.641106 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c80c383-591a-47e0-ad10-993832ccf1be None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:12:56.642514 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 534/1068] 10.222.0.36 () {54 vars in 962 bytes} [Wed Oct 12 00:12:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:12:56.648002 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6269c4a-5e88-4608-a889-059d28a7d38f None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:12:56.648373 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6269c4a-5e88-4608-a889-059d28a7d38f None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:12:56.648714 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6269c4a-5e88-4608-a889-059d28a7d38f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:12:56.697949 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:12:56.728017 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-d6269c4a-5e88-4608-a889-059d28a7d38f None None] MFA Rules not processed for user `59b883abbb674b3c976bacb64194a66d`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:12:56.729227 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-d6269c4a-5e88-4608-a889-059d28a7d38f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:12:56.803884 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 535/1069] 10.222.0.36 () {58 vars in 1059 bytes} [Wed Oct 12 00:12:56 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:12:56.812813 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c80c383-591a-47e0-ad10-993832ccf1be None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:12:56.814547 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-2c80c383-591a-47e0-ad10-993832ccf1be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:12:56.916811 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c80c383-591a-47e0-ad10-993832ccf1be None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:12:56.954846 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:12:56.990333 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bff044-1cde-45c4-9a9c-889ada6f9315 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59b883abbb674b3c976bacb64194a66d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:12:56.990966 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bff044-1cde-45c4-9a9c-889ada6f9315 service glance] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:12:56.991240 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bff044-1cde-45c4-9a9c-889ada6f9315 service glance] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:12:56.991543 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bff044-1cde-45c4-9a9c-889ada6f9315 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:12:56.993128 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-e4bff044-1cde-45c4-9a9c-889ada6f9315 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:12:57.101893 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bff044-1cde-45c4-9a9c-889ada6f9315 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:12:57.103329 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bff044-1cde-45c4-9a9c-889ada6f9315 service glance] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:12:57.121446 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 535/1070] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:12:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:12:57.970092 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c3c39e-3371-4934-9b0a-2df08a66ac57 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:12:57.970655 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c3c39e-3371-4934-9b0a-2df08a66ac57 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:12:57.970655 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c3c39e-3371-4934-9b0a-2df08a66ac57 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:12:57.972052 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 536/1071] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:12:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:12:57.977153 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65768a59-e20f-4f09-b13e-03c64ec4e515 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:12:57.977581 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65768a59-e20f-4f09-b13e-03c64ec4e515 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:12:57.977714 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65768a59-e20f-4f09-b13e-03c64ec4e515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:12:58.034210 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02977760-b972-4fcf-9bfd-f5835db5fa27 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:12:58.034554 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02977760-b972-4fcf-9bfd-f5835db5fa27 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:12:58.043478 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02977760-b972-4fcf-9bfd-f5835db5fa27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:12:58.049836 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-65768a59-e20f-4f09-b13e-03c64ec4e515 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:12:58.051055 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-65768a59-e20f-4f09-b13e-03c64ec4e515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:12:58.069105 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:12:58.105446 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-02977760-b972-4fcf-9bfd-f5835db5fa27 None None] MFA Rules not processed for user `d030d18b595144f19d6ad01d4cd5d52b`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:12:58.106869 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-02977760-b972-4fcf-9bfd-f5835db5fa27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:12:58.150426 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 536/1072] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:12:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:12:58.185878 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:12:58.193306 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 537/1073] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Oct 12 00:12:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 00:12:58.236439 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d313d20d-1db9-4e85-acb6-ec019cfe34c9 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:12:58.236729 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d313d20d-1db9-4e85-acb6-ec019cfe34c9 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:12:58.237008 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d313d20d-1db9-4e85-acb6-ec019cfe34c9 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:12:58.238359 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 537/1074] 10.222.0.36 () {54 vars in 943 bytes} [Wed Oct 12 00:12:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 00:12:58.244988 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1959c299-75b1-4bf2-9335-5d6aa9fcb4aa None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:12:58.245266 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1959c299-75b1-4bf2-9335-5d6aa9fcb4aa None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:12:58.245541 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1959c299-75b1-4bf2-9335-5d6aa9fcb4aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:12:58.333219 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-1959c299-75b1-4bf2-9335-5d6aa9fcb4aa None None] MFA Rules not processed for user `fc1bcffc78844dcbb1a9b7c80285249b`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:12:58.334815 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-1959c299-75b1-4bf2-9335-5d6aa9fcb4aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:12:58.441912 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d313d20d-1db9-4e85-acb6-ec019cfe34c9 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:12:58.443540 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-d313d20d-1db9-4e85-acb6-ec019cfe34c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:12:58.452116 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 538/1075] 10.222.0.36 () {58 vars in 1040 bytes} [Wed Oct 12 00:12:58 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 00:12:58.462123 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1959c299-75b1-4bf2-9335-5d6aa9fcb4aa None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:12:58.472260 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1959c299-75b1-4bf2-9335-5d6aa9fcb4aa None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:12:58.522175 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:12:58.583830 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf8e807-900c-4907-8260-b6e2643da144 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc1bcffc78844dcbb1a9b7c80285249b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:12:58.584443 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf8e807-900c-4907-8260-b6e2643da144 service swift] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:12:58.584720 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf8e807-900c-4907-8260-b6e2643da144 service swift] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:12:58.584986 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf8e807-900c-4907-8260-b6e2643da144 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:12:58.586620 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-3bf8e807-900c-4907-8260-b6e2643da144 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:12:58.601179 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d313d20d-1db9-4e85-acb6-ec019cfe34c9 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:12:58.604104 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:12:58.669431 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf8e807-900c-4907-8260-b6e2643da144 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:12:58.671619 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf8e807-900c-4907-8260-b6e2643da144 service swift] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:12:58.679654 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 539/1076] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Oct 12 00:12:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 12 00:12:58.723030 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d0a651-fd3f-4f49-9964-854298e36e20 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a515b75aa3944912ad2364ffd7924f40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:12:58.723675 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d0a651-fd3f-4f49-9964-854298e36e20 service nova] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:12:58.723948 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d0a651-fd3f-4f49-9964-854298e36e20 service nova] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:12:58.724211 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d0a651-fd3f-4f49-9964-854298e36e20 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:12:58.725870 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-b1d0a651-fd3f-4f49-9964-854298e36e20 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:12:58.815450 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d0a651-fd3f-4f49-9964-854298e36e20 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:12:58.817034 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d0a651-fd3f-4f49-9964-854298e36e20 service nova] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:12:58.834812 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 538/1077] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:12:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 00:13:01.155825 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f312f-1571-4497-99aa-fd17c23909c5 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:13:01.156485 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f312f-1571-4497-99aa-fd17c23909c5 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:13:01.156485 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f312f-1571-4497-99aa-fd17c23909c5 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:13:01.157781 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 540/1078] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:13:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:13:01.163160 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b8c4b1-5f43-4aa4-93d3-4718d4e09007 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:13:01.163656 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b8c4b1-5f43-4aa4-93d3-4718d4e09007 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:13:01.163765 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b8c4b1-5f43-4aa4-93d3-4718d4e09007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:13:01.175690 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:13:01.240821 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-d0b8c4b1-5f43-4aa4-93d3-4718d4e09007 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:13:01.242459 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-d0b8c4b1-5f43-4aa4-93d3-4718d4e09007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:13:01.355287 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 539/1079] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:13:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:13:01.650580 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673f312f-1571-4497-99aa-fd17c23909c5 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:13:01.652212 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-673f312f-1571-4497-99aa-fd17c23909c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:13:01.761062 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:13:01.832846 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673f312f-1571-4497-99aa-fd17c23909c5 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:13:01.959484 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f57a4b2-3c9a-41ad-b75c-07f96e292f97 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a515b75aa3944912ad2364ffd7924f40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:13:01.963595 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f57a4b2-3c9a-41ad-b75c-07f96e292f97 service nova] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:13:01.963882 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f57a4b2-3c9a-41ad-b75c-07f96e292f97 service nova] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:13:01.964152 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f57a4b2-3c9a-41ad-b75c-07f96e292f97 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:13:01.966007 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-0f57a4b2-3c9a-41ad-b75c-07f96e292f97 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:13:02.010212 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:13:02.067790 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f57a4b2-3c9a-41ad-b75c-07f96e292f97 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:13:02.069689 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f57a4b2-3c9a-41ad-b75c-07f96e292f97 service nova] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:13:02.099566 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 541/1080] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:13:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:13:04.459235 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94060870-e37e-41e2-a4e5-cbda6a813ebc None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:13:04.459825 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94060870-e37e-41e2-a4e5-cbda6a813ebc None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:13:04.459825 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94060870-e37e-41e2-a4e5-cbda6a813ebc None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:13:04.461592 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 540/1081] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:13:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:13:04.467232 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603154f5-2a96-4ea1-9d8a-a8c606b7e49a None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:13:04.467683 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603154f5-2a96-4ea1-9d8a-a8c606b7e49a None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:13:04.467859 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603154f5-2a96-4ea1-9d8a-a8c606b7e49a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:13:04.544509 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-603154f5-2a96-4ea1-9d8a-a8c606b7e49a None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:13:04.545752 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-603154f5-2a96-4ea1-9d8a-a8c606b7e49a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:13:04.655423 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 542/1082] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:13:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:13:04.951687 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94060870-e37e-41e2-a4e5-cbda6a813ebc None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:13:04.953251 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-94060870-e37e-41e2-a4e5-cbda6a813ebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:13:05.048760 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:13:05.118494 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94060870-e37e-41e2-a4e5-cbda6a813ebc None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:13:05.247128 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36358e59-5848-4229-bf8a-0e09f217221d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a515b75aa3944912ad2364ffd7924f40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:13:05.247812 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36358e59-5848-4229-bf8a-0e09f217221d service nova] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:13:05.248083 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36358e59-5848-4229-bf8a-0e09f217221d service nova] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:13:05.248345 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36358e59-5848-4229-bf8a-0e09f217221d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:13:05.249928 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-36358e59-5848-4229-bf8a-0e09f217221d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:13:05.305415 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:13:05.350255 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36358e59-5848-4229-bf8a-0e09f217221d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:13:05.351910 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36358e59-5848-4229-bf8a-0e09f217221d service nova] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:13:05.372244 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 541/1083] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:13:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:13:07.710886 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04327b3e-299f-4407-92dc-8237f1d8687f None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:13:07.711475 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04327b3e-299f-4407-92dc-8237f1d8687f None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:13:07.711475 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04327b3e-299f-4407-92dc-8237f1d8687f None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:13:07.712907 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 543/1084] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:13:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:13:07.718323 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bfb824-7ee6-4f99-a7cf-d94fc98d6e77 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:13:07.718914 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bfb824-7ee6-4f99-a7cf-d94fc98d6e77 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:13:07.718914 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bfb824-7ee6-4f99-a7cf-d94fc98d6e77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:13:07.793231 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-82bfb824-7ee6-4f99-a7cf-d94fc98d6e77 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:13:07.795286 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-82bfb824-7ee6-4f99-a7cf-d94fc98d6e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:13:07.913755 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 542/1085] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:13:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:13:08.205610 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04327b3e-299f-4407-92dc-8237f1d8687f None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:13:08.207202 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-04327b3e-299f-4407-92dc-8237f1d8687f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:13:08.228609 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:13:08.370248 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04327b3e-299f-4407-92dc-8237f1d8687f None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:13:08.470671 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:13:08.495217 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b4cd21-0f93-48b5-8f69-015f2b8453e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a515b75aa3944912ad2364ffd7924f40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:13:08.495947 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b4cd21-0f93-48b5-8f69-015f2b8453e9 service nova] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:13:08.496227 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b4cd21-0f93-48b5-8f69-015f2b8453e9 service nova] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:13:08.496494 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b4cd21-0f93-48b5-8f69-015f2b8453e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:13:08.498757 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-77b4cd21-0f93-48b5-8f69-015f2b8453e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:13:08.601994 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b4cd21-0f93-48b5-8f69-015f2b8453e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:13:08.603736 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b4cd21-0f93-48b5-8f69-015f2b8453e9 service nova] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:13:08.625329 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 544/1086] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:13:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:13:17.913884 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=89 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:13:18.622749 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=54 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:13:22.593968 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9df5e3-880e-426b-91f1-c1f18b3459c6 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:13:22.594484 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9df5e3-880e-426b-91f1-c1f18b3459c6 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:13:22.594601 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9df5e3-880e-426b-91f1-c1f18b3459c6 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:13:22.595900 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 543/1087] 10.222.0.36 () {54 vars in 918 bytes} [Wed Oct 12 00:13:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:13:22.600867 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ed4c4-641c-48c6-a723-2d1427e204c0 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:13:22.601270 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ed4c4-641c-48c6-a723-2d1427e204c0 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:13:22.601401 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ed4c4-641c-48c6-a723-2d1427e204c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:13:22.677144 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-df6ed4c4-641c-48c6-a723-2d1427e204c0 None None] MFA Rules not processed for user `1e528da752ed4c78bc747215909f6569`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:13:22.678342 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-df6ed4c4-641c-48c6-a723-2d1427e204c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:13:22.782936 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 545/1088] 10.222.0.36 () {58 vars in 1015 bytes} [Wed Oct 12 00:13:22 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 00:13:22.798261 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b0407-ab26-4417-974d-df5927b27bcd None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:13:22.798542 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b0407-ab26-4417-974d-df5927b27bcd None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:13:22.798899 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b0407-ab26-4417-974d-df5927b27bcd None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:13:22.800375 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 544/1089] 10.222.0.36 () {54 vars in 918 bytes} [Wed Oct 12 00:13:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:13:22.805932 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13b267d-949e-419a-9f0a-4fe28ccc9333 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:13:22.806212 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13b267d-949e-419a-9f0a-4fe28ccc9333 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:13:22.806488 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13b267d-949e-419a-9f0a-4fe28ccc9333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:13:22.881915 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:13:22.884467 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-e13b267d-949e-419a-9f0a-4fe28ccc9333 None None] MFA Rules not processed for user `a515b75aa3944912ad2364ffd7924f40`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:13:22.885677 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-e13b267d-949e-419a-9f0a-4fe28ccc9333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:13:22.999734 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 546/1090] 10.222.0.36 () {58 vars in 1015 bytes} [Wed Oct 12 00:13:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:13:25.565634 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b025f9-0a94-41b0-b6df-bd88df5249e3 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:13:25.566143 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b025f9-0a94-41b0-b6df-bd88df5249e3 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:13:25.566261 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b025f9-0a94-41b0-b6df-bd88df5249e3 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:13:25.567602 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 545/1091] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 12 00:13:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:13:25.572957 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b5be90-5c82-4540-b522-ea4427585127 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:13:25.573364 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b5be90-5c82-4540-b522-ea4427585127 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:13:25.573601 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b5be90-5c82-4540-b522-ea4427585127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:13:25.639739 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-76b5be90-5c82-4540-b522-ea4427585127 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:13:25.641198 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-76b5be90-5c82-4540-b522-ea4427585127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:13:25.694492 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:13:25.735825 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 547/1092] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 12 00:13:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:13:26.030623 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b025f9-0a94-41b0-b6df-bd88df5249e3 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:13:26.032178 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-55b025f9-0a94-41b0-b6df-bd88df5249e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:13:26.177414 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b025f9-0a94-41b0-b6df-bd88df5249e3 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:13:26.249409 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:13:26.288727 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a35903d-26fa-4c68-820d-14e5baf7d0ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a515b75aa3944912ad2364ffd7924f40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:13:26.289259 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a35903d-26fa-4c68-820d-14e5baf7d0ce service nova] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:13:26.289529 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a35903d-26fa-4c68-820d-14e5baf7d0ce service nova] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:13:26.289794 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a35903d-26fa-4c68-820d-14e5baf7d0ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:13:26.291385 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-9a35903d-26fa-4c68-820d-14e5baf7d0ce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:13:26.378893 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a35903d-26fa-4c68-820d-14e5baf7d0ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:13:26.380522 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a35903d-26fa-4c68-820d-14e5baf7d0ce service nova] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:13:26.398141 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 546/1093] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:13:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:13:27.523580 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a8145-a4b2-4c1e-8849-71d6e370fd49 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:13:27.524157 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a8145-a4b2-4c1e-8849-71d6e370fd49 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:13:27.524157 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a8145-a4b2-4c1e-8849-71d6e370fd49 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:13:27.525448 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 548/1094] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 12 00:13:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:13:27.531647 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c6161e-ee1a-4e26-9b3c-0c989604db22 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:13:27.532081 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c6161e-ee1a-4e26-9b3c-0c989604db22 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:13:27.532188 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c6161e-ee1a-4e26-9b3c-0c989604db22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:13:27.609071 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-36c6161e-ee1a-4e26-9b3c-0c989604db22 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:13:27.610286 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-36c6161e-ee1a-4e26-9b3c-0c989604db22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:13:27.637039 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:13:27.704862 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 547/1095] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 12 00:13:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:13:27.853861 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9a8145-a4b2-4c1e-8849-71d6e370fd49 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:13:27.855437 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-da9a8145-a4b2-4c1e-8849-71d6e370fd49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:13:27.997554 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9a8145-a4b2-4c1e-8849-71d6e370fd49 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:13:28.036624 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:13:28.109955 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de075f4-ed2f-4427-a996-45ccf102259a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a515b75aa3944912ad2364ffd7924f40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:13:28.110483 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de075f4-ed2f-4427-a996-45ccf102259a service nova] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:13:28.110756 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de075f4-ed2f-4427-a996-45ccf102259a service nova] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:13:28.111020 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de075f4-ed2f-4427-a996-45ccf102259a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:13:28.112640 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-8de075f4-ed2f-4427-a996-45ccf102259a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:13:28.199701 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de075f4-ed2f-4427-a996-45ccf102259a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:13:28.201295 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de075f4-ed2f-4427-a996-45ccf102259a service nova] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:13:28.219260 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 549/1096] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:13:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:13:29.338402 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48dac09-7215-4c11-82f0-54852193c282 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:13:29.338953 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48dac09-7215-4c11-82f0-54852193c282 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:13:29.338953 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48dac09-7215-4c11-82f0-54852193c282 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:13:29.340517 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 548/1097] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 12 00:13:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:13:29.346372 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67435bae-fba9-4e69-9f4a-fce560dead92 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:13:29.346809 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67435bae-fba9-4e69-9f4a-fce560dead92 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:13:29.346912 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67435bae-fba9-4e69-9f4a-fce560dead92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:13:29.400891 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:13:29.422333 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-67435bae-fba9-4e69-9f4a-fce560dead92 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:13:29.423569 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-67435bae-fba9-4e69-9f4a-fce560dead92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:13:29.517352 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 550/1098] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 12 00:13:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:13:29.673507 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48dac09-7215-4c11-82f0-54852193c282 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:13:29.673507 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-c48dac09-7215-4c11-82f0-54852193c282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:13:29.813290 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48dac09-7215-4c11-82f0-54852193c282 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:13:29.825282 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:13:29.927152 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c4a064-157f-41e6-a1f4-ed2b6c59ace9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a515b75aa3944912ad2364ffd7924f40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:13:29.927716 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c4a064-157f-41e6-a1f4-ed2b6c59ace9 service nova] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:13:29.927996 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c4a064-157f-41e6-a1f4-ed2b6c59ace9 service nova] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:13:29.928265 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c4a064-157f-41e6-a1f4-ed2b6c59ace9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:13:29.929857 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-b5c4a064-157f-41e6-a1f4-ed2b6c59ace9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:13:30.018773 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c4a064-157f-41e6-a1f4-ed2b6c59ace9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:13:30.020394 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c4a064-157f-41e6-a1f4-ed2b6c59ace9 service nova] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:13:30.038419 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 549/1099] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:13:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:13:31.135427 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38b5cbc-7182-4781-a77e-515c0ce2ff6e None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:13:31.135968 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38b5cbc-7182-4781-a77e-515c0ce2ff6e None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:13:31.135968 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38b5cbc-7182-4781-a77e-515c0ce2ff6e None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:13:31.137262 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 551/1100] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 12 00:13:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:13:31.143537 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f1795c-d4ad-4b21-8dbb-5aaa41f737f5 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:13:31.143947 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f1795c-d4ad-4b21-8dbb-5aaa41f737f5 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:13:31.144079 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f1795c-d4ad-4b21-8dbb-5aaa41f737f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:13:31.164316 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:13:31.221907 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-d5f1795c-d4ad-4b21-8dbb-5aaa41f737f5 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:13:31.223122 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-d5f1795c-d4ad-4b21-8dbb-5aaa41f737f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:13:31.317910 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 550/1101] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 12 00:13:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:13:31.469141 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38b5cbc-7182-4781-a77e-515c0ce2ff6e None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:13:31.470581 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-c38b5cbc-7182-4781-a77e-515c0ce2ff6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:13:31.579531 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:13:31.615388 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38b5cbc-7182-4781-a77e-515c0ce2ff6e None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:13:31.729983 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab20bfa4-691c-45e2-8271-33e4b9da35a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a515b75aa3944912ad2364ffd7924f40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:13:31.730515 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab20bfa4-691c-45e2-8271-33e4b9da35a0 service nova] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:13:31.730793 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab20bfa4-691c-45e2-8271-33e4b9da35a0 service nova] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:13:31.731062 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab20bfa4-691c-45e2-8271-33e4b9da35a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:13:31.732689 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-ab20bfa4-691c-45e2-8271-33e4b9da35a0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:13:31.806703 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:13:31.820388 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab20bfa4-691c-45e2-8271-33e4b9da35a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:13:31.831428 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab20bfa4-691c-45e2-8271-33e4b9da35a0 service nova] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:13:31.840268 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 552/1102] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:13:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:13:32.959639 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49b6e68-e971-4cb3-8809-192a6c6f9758 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:13:32.960214 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49b6e68-e971-4cb3-8809-192a6c6f9758 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:13:32.960214 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49b6e68-e971-4cb3-8809-192a6c6f9758 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:13:32.961560 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 551/1103] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 12 00:13:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:13:32.968497 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2ac77a-8636-4fad-847d-708ef2e28acd None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:13:32.968953 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2ac77a-8636-4fad-847d-708ef2e28acd None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:13:32.969100 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2ac77a-8636-4fad-847d-708ef2e28acd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:13:33.043257 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-ce2ac77a-8636-4fad-847d-708ef2e28acd None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:13:33.044512 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-ce2ac77a-8636-4fad-847d-708ef2e28acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:13:33.137658 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 553/1104] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 12 00:13:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:13:33.288052 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49b6e68-e971-4cb3-8809-192a6c6f9758 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:13:33.289610 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-a49b6e68-e971-4cb3-8809-192a6c6f9758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:13:33.374730 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:13:33.435487 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49b6e68-e971-4cb3-8809-192a6c6f9758 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:13:33.545031 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7be301-2a60-4d69-a1b8-2fb497cccad1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a515b75aa3944912ad2364ffd7924f40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:13:33.545633 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7be301-2a60-4d69-a1b8-2fb497cccad1 service nova] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:13:33.545912 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7be301-2a60-4d69-a1b8-2fb497cccad1 service nova] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:13:33.546173 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7be301-2a60-4d69-a1b8-2fb497cccad1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:13:33.547772 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-3f7be301-2a60-4d69-a1b8-2fb497cccad1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:13:33.596522 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:13:33.636892 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7be301-2a60-4d69-a1b8-2fb497cccad1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:13:33.638461 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7be301-2a60-4d69-a1b8-2fb497cccad1 service nova] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:13:33.656004 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 552/1105] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:13:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:13:34.732586 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0af388b-eace-4e17-b585-af77fa001c98 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:13:34.733118 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0af388b-eace-4e17-b585-af77fa001c98 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:13:34.733118 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0af388b-eace-4e17-b585-af77fa001c98 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:13:34.734518 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 554/1106] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 12 00:13:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:13:34.740584 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e2cbdb-c358-42c2-a76f-a348f9a5f8c9 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:13:34.741024 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e2cbdb-c358-42c2-a76f-a348f9a5f8c9 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:13:34.741129 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e2cbdb-c358-42c2-a76f-a348f9a5f8c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:13:34.817343 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-36e2cbdb-c358-42c2-a76f-a348f9a5f8c9 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:13:34.818488 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-36e2cbdb-c358-42c2-a76f-a348f9a5f8c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:13:34.914441 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 553/1107] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 12 00:13:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 00:13:35.062610 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0af388b-eace-4e17-b585-af77fa001c98 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:13:35.064240 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-a0af388b-eace-4e17-b585-af77fa001c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:13:35.112852 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:13:35.209037 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0af388b-eace-4e17-b585-af77fa001c98 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:13:35.320690 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2560a6-781f-41ba-a5e6-2954a6a837f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a515b75aa3944912ad2364ffd7924f40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:13:35.321285 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2560a6-781f-41ba-a5e6-2954a6a837f8 service nova] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:13:35.321558 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2560a6-781f-41ba-a5e6-2954a6a837f8 service nova] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:13:35.321822 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2560a6-781f-41ba-a5e6-2954a6a837f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:13:35.323443 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-9f2560a6-781f-41ba-a5e6-2954a6a837f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:13:35.330371 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:13:35.411149 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2560a6-781f-41ba-a5e6-2954a6a837f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:13:35.412715 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2560a6-781f-41ba-a5e6-2954a6a837f8 service nova] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:13:35.430624 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 555/1108] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:13:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:13:36.506760 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9589cb7e-e1e5-436c-a1b7-9c1667d3e998 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:13:36.507320 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9589cb7e-e1e5-436c-a1b7-9c1667d3e998 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:13:36.507320 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9589cb7e-e1e5-436c-a1b7-9c1667d3e998 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:13:36.508670 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 554/1109] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 12 00:13:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:13:36.515610 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abd8017-e59b-4443-b7d3-1c4b70099433 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:13:36.516072 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abd8017-e59b-4443-b7d3-1c4b70099433 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:13:36.516219 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abd8017-e59b-4443-b7d3-1c4b70099433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:13:36.591530 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-9abd8017-e59b-4443-b7d3-1c4b70099433 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:13:36.592740 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-9abd8017-e59b-4443-b7d3-1c4b70099433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:13:36.658660 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:13:36.687389 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 556/1110] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 12 00:13:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:13:36.835236 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9589cb7e-e1e5-436c-a1b7-9c1667d3e998 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:13:36.836821 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-9589cb7e-e1e5-436c-a1b7-9c1667d3e998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:13:36.854470 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:13:36.979884 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9589cb7e-e1e5-436c-a1b7-9c1667d3e998 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:13:37.064960 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:13:37.089425 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9415ee9-37ee-44ed-b024-5c39b48f3197 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a515b75aa3944912ad2364ffd7924f40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:13:37.089965 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9415ee9-37ee-44ed-b024-5c39b48f3197 service nova] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:13:37.090725 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9415ee9-37ee-44ed-b024-5c39b48f3197 service nova] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:13:37.090990 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9415ee9-37ee-44ed-b024-5c39b48f3197 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:13:37.093368 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-b9415ee9-37ee-44ed-b024-5c39b48f3197 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:13:37.181314 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9415ee9-37ee-44ed-b024-5c39b48f3197 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:13:37.182896 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9415ee9-37ee-44ed-b024-5c39b48f3197 service nova] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:13:37.200355 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 555/1111] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:13:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:13:38.319243 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129b654e-d86e-4487-97e8-d3b642aefb13 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:13:38.319868 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129b654e-d86e-4487-97e8-d3b642aefb13 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:13:38.319868 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129b654e-d86e-4487-97e8-d3b642aefb13 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:13:38.321166 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 557/1112] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 12 00:13:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:13:38.327908 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d935c2-badb-43a4-9a8a-c2ad2501bfe0 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:13:38.328362 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d935c2-badb-43a4-9a8a-c2ad2501bfe0 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:13:38.328475 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d935c2-badb-43a4-9a8a-c2ad2501bfe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:13:38.404811 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-b5d935c2-badb-43a4-9a8a-c2ad2501bfe0 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:13:38.406062 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-b5d935c2-badb-43a4-9a8a-c2ad2501bfe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:13:38.445929 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:13:38.499807 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 556/1113] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 12 00:13:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:13:38.648250 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129b654e-d86e-4487-97e8-d3b642aefb13 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:13:38.649815 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-129b654e-d86e-4487-97e8-d3b642aefb13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:13:38.792874 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129b654e-d86e-4487-97e8-d3b642aefb13 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:13:38.847025 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:13:38.905439 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdb22d2-7f25-4caf-b379-6a82af2b0e63 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a515b75aa3944912ad2364ffd7924f40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:13:38.905976 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdb22d2-7f25-4caf-b379-6a82af2b0e63 service nova] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:13:38.906256 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdb22d2-7f25-4caf-b379-6a82af2b0e63 service nova] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:13:38.906523 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdb22d2-7f25-4caf-b379-6a82af2b0e63 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:13:38.908176 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-acdb22d2-7f25-4caf-b379-6a82af2b0e63 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:13:38.996850 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdb22d2-7f25-4caf-b379-6a82af2b0e63 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:13:38.998450 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdb22d2-7f25-4caf-b379-6a82af2b0e63 service nova] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:13:39.016236 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 558/1114] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:13:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:13:40.112090 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbe80e0-2c56-45db-8ec9-0aa26bb793c8 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:13:40.112659 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbe80e0-2c56-45db-8ec9-0aa26bb793c8 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:13:40.112659 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbe80e0-2c56-45db-8ec9-0aa26bb793c8 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:13:40.113962 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 557/1115] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 12 00:13:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:13:40.120065 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a09f79e-7d32-44c4-b252-91fa39ee9bd3 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:13:40.120491 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a09f79e-7d32-44c4-b252-91fa39ee9bd3 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:13:40.120608 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a09f79e-7d32-44c4-b252-91fa39ee9bd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:13:40.194744 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-9a09f79e-7d32-44c4-b252-91fa39ee9bd3 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:13:40.195966 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-9a09f79e-7d32-44c4-b252-91fa39ee9bd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:13:40.197950 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:13:40.290858 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 559/1116] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 12 00:13:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:13:40.439872 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbe80e0-2c56-45db-8ec9-0aa26bb793c8 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:13:40.441327 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-cdbe80e0-2c56-45db-8ec9-0aa26bb793c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:13:40.585342 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbe80e0-2c56-45db-8ec9-0aa26bb793c8 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:13:40.612172 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:13:40.696520 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd095c13-5190-4762-bac6-a608f7525967 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a515b75aa3944912ad2364ffd7924f40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:13:40.697044 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd095c13-5190-4762-bac6-a608f7525967 service nova] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:13:40.697317 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd095c13-5190-4762-bac6-a608f7525967 service nova] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:13:40.697584 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd095c13-5190-4762-bac6-a608f7525967 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:13:40.699169 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-cd095c13-5190-4762-bac6-a608f7525967 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:13:40.787905 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd095c13-5190-4762-bac6-a608f7525967 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:13:40.789476 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd095c13-5190-4762-bac6-a608f7525967 service nova] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:13:40.806970 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 558/1117] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:13:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:13:42.843495 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839677ae-8ce5-46b6-8c0c-a0d4f54b61ee None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:13:42.843958 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839677ae-8ce5-46b6-8c0c-a0d4f54b61ee None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:13:42.844267 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839677ae-8ce5-46b6-8c0c-a0d4f54b61ee None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:13:42.845392 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 560/1118] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 12 00:13:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:13:42.850908 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f886b5d1-3446-4008-ac4c-f90f77a82b0b None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:13:42.851317 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f886b5d1-3446-4008-ac4c-f90f77a82b0b None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:13:42.851495 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f886b5d1-3446-4008-ac4c-f90f77a82b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:13:42.891707 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:13:42.917572 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-f886b5d1-3446-4008-ac4c-f90f77a82b0b None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:13:42.918787 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-f886b5d1-3446-4008-ac4c-f90f77a82b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:13:43.016835 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 559/1119] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 12 00:13:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:13:43.297913 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839677ae-8ce5-46b6-8c0c-a0d4f54b61ee None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:13:43.299481 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-839677ae-8ce5-46b6-8c0c-a0d4f54b61ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:13:43.404763 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839677ae-8ce5-46b6-8c0c-a0d4f54b61ee None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:13:43.437774 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:13:43.477190 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70176830-f433-4473-bfdc-07b58e119e89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:13:43.477734 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70176830-f433-4473-bfdc-07b58e119e89 service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:13:43.478013 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70176830-f433-4473-bfdc-07b58e119e89 service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:13:43.478280 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70176830-f433-4473-bfdc-07b58e119e89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:13:43.480015 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-70176830-f433-4473-bfdc-07b58e119e89 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:13:43.588787 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70176830-f433-4473-bfdc-07b58e119e89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:13:43.590276 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70176830-f433-4473-bfdc-07b58e119e89 service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:13:43.608282 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 561/1120] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:13:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:13:45.889248 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34476718-4b98-4a21-8c53-8c98439192ca None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:13:45.889817 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34476718-4b98-4a21-8c53-8c98439192ca None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:13:45.889817 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34476718-4b98-4a21-8c53-8c98439192ca None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:13:45.891504 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 560/1121] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 12 00:13:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:13:45.896797 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe18f81-d55f-48f1-9bbe-fb1c74025e00 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:13:45.897193 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe18f81-d55f-48f1-9bbe-fb1c74025e00 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:13:45.897339 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe18f81-d55f-48f1-9bbe-fb1c74025e00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:13:45.962884 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-cbe18f81-d55f-48f1-9bbe-fb1c74025e00 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:13:45.964136 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-cbe18f81-d55f-48f1-9bbe-fb1c74025e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:13:45.977271 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:13:46.060246 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 562/1122] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 12 00:13:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:13:46.344030 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34476718-4b98-4a21-8c53-8c98439192ca None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:13:46.345623 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-34476718-4b98-4a21-8c53-8c98439192ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:13:46.446674 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34476718-4b98-4a21-8c53-8c98439192ca None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:13:46.459874 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:13:46.520281 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa95957-01fd-46a1-b875-8c65693b2584 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:13:46.520883 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa95957-01fd-46a1-b875-8c65693b2584 service neutron] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:13:46.521159 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa95957-01fd-46a1-b875-8c65693b2584 service neutron] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:13:46.521428 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa95957-01fd-46a1-b875-8c65693b2584 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:13:46.523016 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-efa95957-01fd-46a1-b875-8c65693b2584 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:13:46.631802 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa95957-01fd-46a1-b875-8c65693b2584 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:13:46.633166 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa95957-01fd-46a1-b875-8c65693b2584 service neutron] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:13:46.651826 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 561/1123] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:13:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:13:49.267580 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42446a5-b729-4111-996b-11b9d24bae1b None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:13:49.268111 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42446a5-b729-4111-996b-11b9d24bae1b None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:13:49.268111 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42446a5-b729-4111-996b-11b9d24bae1b None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:13:49.269364 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 563/1124] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 12 00:13:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:13:49.274543 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2658eace-3904-4994-bdc3-05e47fa9fd11 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:13:49.274950 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2658eace-3904-4994-bdc3-05e47fa9fd11 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:13:49.275076 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2658eace-3904-4994-bdc3-05e47fa9fd11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:13:49.322138 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:13:49.340445 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-2658eace-3904-4994-bdc3-05e47fa9fd11 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:13:49.341633 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-2658eace-3904-4994-bdc3-05e47fa9fd11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:13:49.435136 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 562/1125] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 12 00:13:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:13:49.719254 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42446a5-b729-4111-996b-11b9d24bae1b None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:13:49.720812 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-e42446a5-b729-4111-996b-11b9d24bae1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:13:49.823733 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42446a5-b729-4111-996b-11b9d24bae1b None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:13:49.863678 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:13:49.897751 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155db3e2-9f42-4d56-adee-de81af74c789 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:13:49.898361 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155db3e2-9f42-4d56-adee-de81af74c789 service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:13:49.898639 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155db3e2-9f42-4d56-adee-de81af74c789 service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:13:49.898911 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155db3e2-9f42-4d56-adee-de81af74c789 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:13:49.900553 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-155db3e2-9f42-4d56-adee-de81af74c789 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:13:50.007077 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155db3e2-9f42-4d56-adee-de81af74c789 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:13:50.008534 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155db3e2-9f42-4d56-adee-de81af74c789 service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:13:50.027095 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 564/1126] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:13:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:13:52.270361 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091582e4-4ccc-48f6-9b0e-9875d5e23355 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:13:52.270904 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091582e4-4ccc-48f6-9b0e-9875d5e23355 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:13:52.270904 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091582e4-4ccc-48f6-9b0e-9875d5e23355 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:13:52.272320 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 563/1127] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 12 00:13:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:13:52.277801 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf68533-9a61-4e43-b859-1160bc0e321c None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:13:52.278180 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf68533-9a61-4e43-b859-1160bc0e321c None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:13:52.278352 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf68533-9a61-4e43-b859-1160bc0e321c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:13:52.344100 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-9cf68533-9a61-4e43-b859-1160bc0e321c None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:13:52.345310 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-9cf68533-9a61-4e43-b859-1160bc0e321c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:13:52.365890 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:13:52.447430 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 565/1128] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 12 00:13:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:13:52.734909 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091582e4-4ccc-48f6-9b0e-9875d5e23355 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:13:52.736529 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-091582e4-4ccc-48f6-9b0e-9875d5e23355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:13:52.844065 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091582e4-4ccc-48f6-9b0e-9875d5e23355 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:13:52.865148 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:13:52.918596 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691df539-b40b-4e5f-8a77-b2fcf2aafa1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:13:52.919214 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691df539-b40b-4e5f-8a77-b2fcf2aafa1e service neutron] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:13:52.919516 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691df539-b40b-4e5f-8a77-b2fcf2aafa1e service neutron] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:13:52.919781 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691df539-b40b-4e5f-8a77-b2fcf2aafa1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:13:52.921373 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-691df539-b40b-4e5f-8a77-b2fcf2aafa1e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:13:53.029685 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691df539-b40b-4e5f-8a77-b2fcf2aafa1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:13:53.031112 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691df539-b40b-4e5f-8a77-b2fcf2aafa1e service neutron] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:13:53.049388 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 564/1129] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:13:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:13:55.223630 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b1477-5e88-4e7b-90cd-756c723d8d50 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:13:55.224182 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b1477-5e88-4e7b-90cd-756c723d8d50 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:13:55.224182 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b1477-5e88-4e7b-90cd-756c723d8d50 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:13:55.225413 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 566/1130] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 12 00:13:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:13:55.230685 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c839e1-75fa-46fd-95a3-f8eb54cdaab4 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:13:55.231138 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c839e1-75fa-46fd-95a3-f8eb54cdaab4 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:13:55.231236 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c839e1-75fa-46fd-95a3-f8eb54cdaab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:13:55.294194 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:13:55.297000 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-15c839e1-75fa-46fd-95a3-f8eb54cdaab4 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:13:55.298203 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-15c839e1-75fa-46fd-95a3-f8eb54cdaab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:13:55.393504 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 565/1131] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 12 00:13:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:13:55.675734 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72b1477-5e88-4e7b-90cd-756c723d8d50 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:13:55.677302 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-e72b1477-5e88-4e7b-90cd-756c723d8d50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:13:55.779122 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72b1477-5e88-4e7b-90cd-756c723d8d50 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:13:55.828350 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:13:55.851931 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ded20b-5875-4ade-ab67-527e0b2db190 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:13:55.852530 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ded20b-5875-4ade-ab67-527e0b2db190 service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:13:55.852806 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ded20b-5875-4ade-ab67-527e0b2db190 service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:13:55.853074 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ded20b-5875-4ade-ab67-527e0b2db190 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:13:55.854673 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-89ded20b-5875-4ade-ab67-527e0b2db190 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:13:55.960446 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ded20b-5875-4ade-ab67-527e0b2db190 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:13:55.961874 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ded20b-5875-4ade-ab67-527e0b2db190 service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:13:55.980682 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 567/1132] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:13:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:13:58.186235 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0d34b7-b595-48ff-a930-8cfeabe941e2 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:13:58.186813 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0d34b7-b595-48ff-a930-8cfeabe941e2 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:13:58.186813 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0d34b7-b595-48ff-a930-8cfeabe941e2 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:13:58.188140 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 566/1133] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 12 00:13:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:13:58.193439 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75307a8-e585-44fb-bb2f-8f50c5e3a205 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:13:58.193846 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75307a8-e585-44fb-bb2f-8f50c5e3a205 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:13:58.194067 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75307a8-e585-44fb-bb2f-8f50c5e3a205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:13:58.258786 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-e75307a8-e585-44fb-bb2f-8f50c5e3a205 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:13:58.260279 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-e75307a8-e585-44fb-bb2f-8f50c5e3a205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:13:58.289870 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:13:58.358257 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 568/1134] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 12 00:13:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:13:58.643774 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0d34b7-b595-48ff-a930-8cfeabe941e2 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:13:58.645315 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-bb0d34b7-b595-48ff-a930-8cfeabe941e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:13:58.747055 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0d34b7-b595-48ff-a930-8cfeabe941e2 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:13:58.775764 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:13:58.821042 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917cb37f-e318-4d50-8e93-4ed06572d574 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:13:58.821570 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917cb37f-e318-4d50-8e93-4ed06572d574 service neutron] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:13:58.821846 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917cb37f-e318-4d50-8e93-4ed06572d574 service neutron] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:13:58.822157 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917cb37f-e318-4d50-8e93-4ed06572d574 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:13:58.824502 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-917cb37f-e318-4d50-8e93-4ed06572d574 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:13:58.932397 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917cb37f-e318-4d50-8e93-4ed06572d574 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:13:58.934771 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917cb37f-e318-4d50-8e93-4ed06572d574 service neutron] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:13:58.952617 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 567/1135] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:13:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:14:01.141603 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20131535-25ca-4573-94d0-bc48bf687d13 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:14:01.142115 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20131535-25ca-4573-94d0-bc48bf687d13 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:14:01.142115 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20131535-25ca-4573-94d0-bc48bf687d13 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:14:01.143441 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 569/1136] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 12 00:14:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:14:01.148648 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3f1b38-9b5e-4971-8a45-23827b75f8be None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:14:01.149037 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3f1b38-9b5e-4971-8a45-23827b75f8be None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:14:01.149206 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3f1b38-9b5e-4971-8a45-23827b75f8be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:14:01.215197 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-1f3f1b38-9b5e-4971-8a45-23827b75f8be None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:14:01.216463 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-1f3f1b38-9b5e-4971-8a45-23827b75f8be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:14:01.224834 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:14:01.312956 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 568/1137] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 12 00:14:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:14:01.597063 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20131535-25ca-4573-94d0-bc48bf687d13 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:14:01.598629 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-20131535-25ca-4573-94d0-bc48bf687d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:14:01.702490 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20131535-25ca-4573-94d0-bc48bf687d13 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:14:01.759506 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:14:01.778019 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a069191-d7f4-4baf-ab15-bb340449a67b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:14:01.778630 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a069191-d7f4-4baf-ab15-bb340449a67b service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:14:01.778910 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a069191-d7f4-4baf-ab15-bb340449a67b service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:14:01.779182 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a069191-d7f4-4baf-ab15-bb340449a67b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:14:01.780820 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-6a069191-d7f4-4baf-ab15-bb340449a67b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:14:01.888592 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a069191-d7f4-4baf-ab15-bb340449a67b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:14:01.890072 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a069191-d7f4-4baf-ab15-bb340449a67b service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:14:01.908721 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 570/1138] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:14:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:14:04.106508 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f882fe5b-f512-4409-afc7-dd22210be6eb None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:14:04.107122 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f882fe5b-f512-4409-afc7-dd22210be6eb None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:14:04.107122 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f882fe5b-f512-4409-afc7-dd22210be6eb None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:14:04.108392 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 569/1139] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 12 00:14:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:14:04.113604 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9cca10-ee3d-4978-b50e-e3fb212a6a7b None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:14:04.114034 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9cca10-ee3d-4978-b50e-e3fb212a6a7b None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:14:04.114147 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9cca10-ee3d-4978-b50e-e3fb212a6a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:14:04.178550 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-7c9cca10-ee3d-4978-b50e-e3fb212a6a7b None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:14:04.179790 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-7c9cca10-ee3d-4978-b50e-e3fb212a6a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:14:04.218190 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:14:04.275414 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 571/1140] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 12 00:14:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:14:04.561504 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f882fe5b-f512-4409-afc7-dd22210be6eb None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:14:04.563051 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-f882fe5b-f512-4409-afc7-dd22210be6eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:14:04.666469 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f882fe5b-f512-4409-afc7-dd22210be6eb None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:14:04.703886 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:14:04.739607 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16f1a0b-9eb5-4fe4-872c-70dfd289e847 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:14:04.740208 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16f1a0b-9eb5-4fe4-872c-70dfd289e847 service neutron] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:14:04.740482 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16f1a0b-9eb5-4fe4-872c-70dfd289e847 service neutron] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:14:04.740752 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16f1a0b-9eb5-4fe4-872c-70dfd289e847 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:14:04.742336 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-c16f1a0b-9eb5-4fe4-872c-70dfd289e847 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:14:04.849169 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16f1a0b-9eb5-4fe4-872c-70dfd289e847 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:14:04.850599 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16f1a0b-9eb5-4fe4-872c-70dfd289e847 service neutron] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:14:04.869769 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 570/1141] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:14:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:14:07.037191 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf7ca95-04a7-4b08-9305-0492d5e8e7b0 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:14:07.037758 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf7ca95-04a7-4b08-9305-0492d5e8e7b0 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:14:07.037758 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf7ca95-04a7-4b08-9305-0492d5e8e7b0 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:14:07.039124 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 572/1142] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 12 00:14:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:14:07.045292 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab1d49-188b-4626-8ab0-1611b31d5b44 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:14:07.045728 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab1d49-188b-4626-8ab0-1611b31d5b44 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:14:07.045944 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab1d49-188b-4626-8ab0-1611b31d5b44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:14:07.112256 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-72ab1d49-188b-4626-8ab0-1611b31d5b44 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:14:07.113381 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-72ab1d49-188b-4626-8ab0-1611b31d5b44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:14:07.130945 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:14:07.207384 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 571/1143] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 12 00:14:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:14:07.489218 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf7ca95-04a7-4b08-9305-0492d5e8e7b0 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:14:07.490759 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-4bf7ca95-04a7-4b08-9305-0492d5e8e7b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:14:07.593512 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf7ca95-04a7-4b08-9305-0492d5e8e7b0 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:14:07.660159 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:14:07.666217 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d506a7-33db-4b3d-8c4b-94fc18451308 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:14:07.666865 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d506a7-33db-4b3d-8c4b-94fc18451308 service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:14:07.667155 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d506a7-33db-4b3d-8c4b-94fc18451308 service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:14:07.667452 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d506a7-33db-4b3d-8c4b-94fc18451308 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:14:07.675812 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-88d506a7-33db-4b3d-8c4b-94fc18451308 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:14:07.776554 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d506a7-33db-4b3d-8c4b-94fc18451308 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:14:07.777902 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d506a7-33db-4b3d-8c4b-94fc18451308 service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:14:07.795194 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 573/1144] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:14:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:14:09.958128 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3b5d23-07f5-44ff-9cdd-81b2ba86d90f None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:14:09.958717 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3b5d23-07f5-44ff-9cdd-81b2ba86d90f None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:14:09.958717 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3b5d23-07f5-44ff-9cdd-81b2ba86d90f None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:14:09.960043 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 572/1145] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 12 00:14:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:14:09.965848 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0a0b59-7bf5-4116-a807-bb7f13d1171f None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:14:09.966289 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0a0b59-7bf5-4116-a807-bb7f13d1171f None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:14:09.966385 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0a0b59-7bf5-4116-a807-bb7f13d1171f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:14:10.031233 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-eb0a0b59-7bf5-4116-a807-bb7f13d1171f None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:14:10.032452 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-eb0a0b59-7bf5-4116-a807-bb7f13d1171f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:14:10.079064 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:14:10.126720 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 574/1146] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 12 00:14:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:14:10.404353 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3b5d23-07f5-44ff-9cdd-81b2ba86d90f None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:14:10.405883 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-ff3b5d23-07f5-44ff-9cdd-81b2ba86d90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:14:10.507389 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3b5d23-07f5-44ff-9cdd-81b2ba86d90f None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:14:10.555332 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:14:10.581452 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7225886b-95d6-427c-9e62-0dffa2391747 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:14:10.582117 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7225886b-95d6-427c-9e62-0dffa2391747 service neutron] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:14:10.582571 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7225886b-95d6-427c-9e62-0dffa2391747 service neutron] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:14:10.583011 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7225886b-95d6-427c-9e62-0dffa2391747 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:14:10.584960 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-7225886b-95d6-427c-9e62-0dffa2391747 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:14:10.690722 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7225886b-95d6-427c-9e62-0dffa2391747 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:14:10.692100 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7225886b-95d6-427c-9e62-0dffa2391747 service neutron] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:14:10.710924 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 573/1147] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:14:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:14:20.125131 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=22 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:14:20.708705 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=66 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:14:30.533835 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd4c7e9-47a7-409f-adec-f3b3474f2dac None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:14:30.534408 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd4c7e9-47a7-409f-adec-f3b3474f2dac None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:14:30.534408 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd4c7e9-47a7-409f-adec-f3b3474f2dac None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:14:30.535741 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 575/1148] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:14:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:14:30.541687 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a8be68-35ea-404a-8113-3a125c6b4cc5 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:14:30.542468 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a8be68-35ea-404a-8113-3a125c6b4cc5 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:14:30.543105 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a8be68-35ea-404a-8113-3a125c6b4cc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:14:30.610272 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-f2a8be68-35ea-404a-8113-3a125c6b4cc5 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:14:30.611550 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-f2a8be68-35ea-404a-8113-3a125c6b4cc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:14:30.708256 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 574/1149] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:14:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:14:31.012503 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a776770-87a7-40aa-a89b-7e7b02ca5437 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:14:31.012777 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a776770-87a7-40aa-a89b-7e7b02ca5437 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:14:31.013043 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a776770-87a7-40aa-a89b-7e7b02ca5437 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:14:31.014364 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 576/1150] 10.222.0.36 () {54 vars in 962 bytes} [Wed Oct 12 00:14:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:14:31.019652 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02af59d5-bc45-47e1-9425-1e3845254dd6 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:14:31.019935 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02af59d5-bc45-47e1-9425-1e3845254dd6 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:14:31.020202 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02af59d5-bc45-47e1-9425-1e3845254dd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:14:31.094878 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-02af59d5-bc45-47e1-9425-1e3845254dd6 None None] MFA Rules not processed for user `59b883abbb674b3c976bacb64194a66d`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:14:31.096143 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-02af59d5-bc45-47e1-9425-1e3845254dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:14:31.116054 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:14:31.171428 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 575/1151] 10.222.0.36 () {58 vars in 1059 bytes} [Wed Oct 12 00:14:31 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:14:31.179738 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a776770-87a7-40aa-a89b-7e7b02ca5437 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:14:31.181314 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-9a776770-87a7-40aa-a89b-7e7b02ca5437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:14:31.283808 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a776770-87a7-40aa-a89b-7e7b02ca5437 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:14:31.357191 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1df58c-5b45-4e17-9838-e30677f511fc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59b883abbb674b3c976bacb64194a66d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:14:31.357787 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1df58c-5b45-4e17-9838-e30677f511fc service glance] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:14:31.358059 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1df58c-5b45-4e17-9838-e30677f511fc service glance] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:14:31.358324 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1df58c-5b45-4e17-9838-e30677f511fc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:14:31.359927 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-ce1df58c-5b45-4e17-9838-e30677f511fc service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:14:31.411901 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:14:31.464750 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1df58c-5b45-4e17-9838-e30677f511fc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:14:31.466202 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1df58c-5b45-4e17-9838-e30677f511fc service glance] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:14:31.484495 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 577/1152] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:14:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:14:33.716414 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f488fed5-5a9c-4fd8-b90e-99b8b69057ff None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:14:33.716971 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f488fed5-5a9c-4fd8-b90e-99b8b69057ff None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:14:33.716971 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f488fed5-5a9c-4fd8-b90e-99b8b69057ff None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:14:33.718295 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 576/1153] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:14:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:14:33.724371 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f76dac-07b6-46a0-9e85-a7aa448f00f7 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:14:33.724814 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f76dac-07b6-46a0-9e85-a7aa448f00f7 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:14:33.724939 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f76dac-07b6-46a0-9e85-a7aa448f00f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:14:33.793762 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-50f76dac-07b6-46a0-9e85-a7aa448f00f7 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:14:33.794974 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-50f76dac-07b6-46a0-9e85-a7aa448f00f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:14:33.895189 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 578/1154] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:14:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:14:34.171018 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f488fed5-5a9c-4fd8-b90e-99b8b69057ff None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:14:34.172600 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-f488fed5-5a9c-4fd8-b90e-99b8b69057ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:14:34.297035 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f488fed5-5a9c-4fd8-b90e-99b8b69057ff None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:14:34.347064 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:14:34.391217 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393ea1f1-d95c-4051-aac0-a1c4a846063c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:14:34.391854 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393ea1f1-d95c-4051-aac0-a1c4a846063c admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:14:34.392129 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393ea1f1-d95c-4051-aac0-a1c4a846063c admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:14:34.392396 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393ea1f1-d95c-4051-aac0-a1c4a846063c admin admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:14:34.393026 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393ea1f1-d95c-4051-aac0-a1c4a846063c admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:14:34.394030 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: WARNING py.warnings [None req-393ea1f1-d95c-4051-aac0-a1c4a846063c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 12 00:14:34.394030 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: warnings.warn(msg) Oct 12 00:14:34.394030 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: Oct 12 00:14:34.395184 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393ea1f1-d95c-4051-aac0-a1c4a846063c admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:14:34.414002 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 577/1155] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Oct 12 00:14:34 2022] GET /identity/v3/projects => generated 2553 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:14:36.545616 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0600ab18-43a6-49b1-9910-26751681c668 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:14:36.546060 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0600ab18-43a6-49b1-9910-26751681c668 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:14:36.546165 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0600ab18-43a6-49b1-9910-26751681c668 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:14:36.547519 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 579/1156] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:14:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:14:36.552720 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6d2e10-24eb-41e7-a6ea-44bd3b178c90 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:14:36.553163 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6d2e10-24eb-41e7-a6ea-44bd3b178c90 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:14:36.553265 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6d2e10-24eb-41e7-a6ea-44bd3b178c90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:14:36.618745 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-8e6d2e10-24eb-41e7-a6ea-44bd3b178c90 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:14:36.619984 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-8e6d2e10-24eb-41e7-a6ea-44bd3b178c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:14:36.714571 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 578/1157] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:14:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:14:37.010902 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0600ab18-43a6-49b1-9910-26751681c668 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:14:37.012477 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-0600ab18-43a6-49b1-9910-26751681c668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:14:37.022405 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:14:37.156061 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0600ab18-43a6-49b1-9910-26751681c668 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:14:37.235433 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:14:37.269492 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffebab62-53a4-4050-9c7e-a3750bda7e43 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a515b75aa3944912ad2364ffd7924f40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:14:37.270101 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffebab62-53a4-4050-9c7e-a3750bda7e43 service nova] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:14:37.270379 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffebab62-53a4-4050-9c7e-a3750bda7e43 service nova] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:14:37.270653 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffebab62-53a4-4050-9c7e-a3750bda7e43 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:14:37.272283 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-ffebab62-53a4-4050-9c7e-a3750bda7e43 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:14:37.359460 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffebab62-53a4-4050-9c7e-a3750bda7e43 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:14:37.361060 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffebab62-53a4-4050-9c7e-a3750bda7e43 service nova] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:14:37.379000 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 580/1158] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:14:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:14:39.686259 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdce619-2ab0-4394-b91a-8e114695dfef None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:14:39.686823 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdce619-2ab0-4394-b91a-8e114695dfef None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:14:39.686823 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdce619-2ab0-4394-b91a-8e114695dfef None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:14:39.688153 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 579/1159] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:14:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:14:39.693489 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cd280a-5a95-442f-9006-ae359cf3bdaa None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:14:39.693906 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cd280a-5a95-442f-9006-ae359cf3bdaa None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:14:39.694020 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cd280a-5a95-442f-9006-ae359cf3bdaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:14:39.759036 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-80cd280a-5a95-442f-9006-ae359cf3bdaa None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:14:39.760255 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-80cd280a-5a95-442f-9006-ae359cf3bdaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:14:39.805232 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:14:39.853570 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 581/1160] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:14:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:14:40.139742 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdce619-2ab0-4394-b91a-8e114695dfef None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:14:40.141559 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-9fdce619-2ab0-4394-b91a-8e114695dfef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:14:40.164492 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:14:40.243215 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdce619-2ab0-4394-b91a-8e114695dfef None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:14:40.316775 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ff20dc-cc75-40de-9f4b-d7cefdf886cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:14:40.317442 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ff20dc-cc75-40de-9f4b-d7cefdf886cd service neutron] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:14:40.317722 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ff20dc-cc75-40de-9f4b-d7cefdf886cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:14:40.317988 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ff20dc-cc75-40de-9f4b-d7cefdf886cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:14:40.319613 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-65ff20dc-cc75-40de-9f4b-d7cefdf886cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:14:40.393835 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:14:40.427179 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ff20dc-cc75-40de-9f4b-d7cefdf886cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:14:40.429129 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ff20dc-cc75-40de-9f4b-d7cefdf886cd service neutron] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:14:40.447428 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 580/1161] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:14:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:14:42.607480 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce7426d-1e4f-442f-bd55-15da5107efe1 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:14:42.608012 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce7426d-1e4f-442f-bd55-15da5107efe1 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:14:42.608012 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce7426d-1e4f-442f-bd55-15da5107efe1 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:14:42.609397 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 582/1162] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:14:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:14:42.614479 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7648055-0c5a-45fd-bb0b-990a7d5b7d34 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:14:42.614934 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7648055-0c5a-45fd-bb0b-990a7d5b7d34 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:14:42.615036 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7648055-0c5a-45fd-bb0b-990a7d5b7d34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:14:42.680337 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-d7648055-0c5a-45fd-bb0b-990a7d5b7d34 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:14:42.681547 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-d7648055-0c5a-45fd-bb0b-990a7d5b7d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:14:42.775203 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 581/1163] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:14:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:14:43.060489 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce7426d-1e4f-442f-bd55-15da5107efe1 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:14:43.062172 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-5ce7426d-1e4f-442f-bd55-15da5107efe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:14:43.163674 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce7426d-1e4f-442f-bd55-15da5107efe1 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:14:43.234494 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:14:43.239114 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31be89be-2ccb-4a5d-a461-6e8534f3ea61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:14:43.239114 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31be89be-2ccb-4a5d-a461-6e8534f3ea61 service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:14:43.239114 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31be89be-2ccb-4a5d-a461-6e8534f3ea61 service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:14:43.239114 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31be89be-2ccb-4a5d-a461-6e8534f3ea61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:14:43.239537 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-31be89be-2ccb-4a5d-a461-6e8534f3ea61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:14:43.348770 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31be89be-2ccb-4a5d-a461-6e8534f3ea61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:14:43.350212 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31be89be-2ccb-4a5d-a461-6e8534f3ea61 service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:14:43.368947 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 583/1164] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:14:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:14:45.590495 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87289498-bd5c-421c-ae6f-395cb8f7fee5 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:14:45.591066 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87289498-bd5c-421c-ae6f-395cb8f7fee5 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:14:45.591066 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87289498-bd5c-421c-ae6f-395cb8f7fee5 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:14:45.592406 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 582/1165] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:14:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:14:45.597628 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92349961-e106-4dd6-87b3-8d6e2aef6c35 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:14:45.598070 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92349961-e106-4dd6-87b3-8d6e2aef6c35 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:14:45.598258 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92349961-e106-4dd6-87b3-8d6e2aef6c35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:14:45.663074 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-92349961-e106-4dd6-87b3-8d6e2aef6c35 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:14:45.664310 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-92349961-e106-4dd6-87b3-8d6e2aef6c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:14:45.715956 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:14:45.757984 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 584/1166] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:14:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:14:46.047327 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87289498-bd5c-421c-ae6f-395cb8f7fee5 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:14:46.048901 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-87289498-bd5c-421c-ae6f-395cb8f7fee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:14:46.085298 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:14:46.152154 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87289498-bd5c-421c-ae6f-395cb8f7fee5 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:14:46.223769 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56765d7-affb-410e-a75f-8bc5c086ea50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:14:46.224365 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56765d7-affb-410e-a75f-8bc5c086ea50 service neutron] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:14:46.224639 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56765d7-affb-410e-a75f-8bc5c086ea50 service neutron] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:14:46.224907 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56765d7-affb-410e-a75f-8bc5c086ea50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:14:46.226490 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-c56765d7-affb-410e-a75f-8bc5c086ea50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:14:46.305373 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:14:46.333967 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56765d7-affb-410e-a75f-8bc5c086ea50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:14:46.335416 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56765d7-affb-410e-a75f-8bc5c086ea50 service neutron] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:14:46.353562 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 583/1167] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:14:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:14:48.902218 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b2215a-8b19-423f-8f52-419f22bb53d4 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:14:48.902715 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b2215a-8b19-423f-8f52-419f22bb53d4 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:14:48.902810 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b2215a-8b19-423f-8f52-419f22bb53d4 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:14:48.904227 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 585/1168] 10.222.0.36 () {54 vars in 927 bytes} [Wed Oct 12 00:14:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:14:48.909723 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc39415-b573-498a-9a54-d1bba282a824 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:14:48.910130 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc39415-b573-498a-9a54-d1bba282a824 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:14:48.910252 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc39415-b573-498a-9a54-d1bba282a824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:14:48.975176 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-6fc39415-b573-498a-9a54-d1bba282a824 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:14:48.976423 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-6fc39415-b573-498a-9a54-d1bba282a824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:14:49.070491 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 584/1169] 10.222.0.36 () {58 vars in 1024 bytes} [Wed Oct 12 00:14:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:14:49.359323 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b2215a-8b19-423f-8f52-419f22bb53d4 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:14:49.360890 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-e1b2215a-8b19-423f-8f52-419f22bb53d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:14:49.463487 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b2215a-8b19-423f-8f52-419f22bb53d4 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:14:49.534243 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8291c6-583b-4d5a-a342-cc6c628ae39d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:14:49.534844 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8291c6-583b-4d5a-a342-cc6c628ae39d service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:14:49.535122 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8291c6-583b-4d5a-a342-cc6c628ae39d service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:14:49.535410 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8291c6-583b-4d5a-a342-cc6c628ae39d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:14:49.537043 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-1b8291c6-583b-4d5a-a342-cc6c628ae39d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:14:49.546714 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:14:49.646737 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8291c6-583b-4d5a-a342-cc6c628ae39d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:14:49.648115 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8291c6-583b-4d5a-a342-cc6c628ae39d service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:14:49.665371 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 586/1170] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:14:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:14:59.069021 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=82 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:14:59.663132 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=50 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:16:30.312049 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b380fac-940d-466a-940e-3abdb3edc99c None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:16:30.312595 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b380fac-940d-466a-940e-3abdb3edc99c None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:16:30.312595 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b380fac-940d-466a-940e-3abdb3edc99c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:16:30.388327 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-4b380fac-940d-466a-940e-3abdb3edc99c None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:16:30.389534 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-4b380fac-940d-466a-940e-3abdb3edc99c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:16:30.483034 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 585/1171] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:16:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:16:30.490893 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf774c4c-0cb4-48ab-8634-e845be1f22e1 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:16:30.491898 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf774c4c-0cb4-48ab-8634-e845be1f22e1 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:16:30.492000 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf774c4c-0cb4-48ab-8634-e845be1f22e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:16:30.568388 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-bf774c4c-0cb4-48ab-8634-e845be1f22e1 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:16:30.569515 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-bf774c4c-0cb4-48ab-8634-e845be1f22e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:16:30.665292 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 587/1172] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:16:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:16:30.669948 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b380fac-940d-466a-940e-3abdb3edc99c None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:16:30.671201 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-4b380fac-940d-466a-940e-3abdb3edc99c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:16:30.683301 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b380fac-940d-466a-940e-3abdb3edc99c None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:16:30.746121 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:16:30.778418 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e03b36-6a6c-4918-b458-10284b6683b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:16:30.779016 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e03b36-6a6c-4918-b458-10284b6683b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:16:30.779293 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e03b36-6a6c-4918-b458-10284b6683b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:16:30.779583 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e03b36-6a6c-4918-b458-10284b6683b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:16:30.780209 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e03b36-6a6c-4918-b458-10284b6683b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:16:30.780505 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e03b36-6a6c-4918-b458-10284b6683b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:16:30.781503 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: WARNING py.warnings [None req-43e03b36-6a6c-4918-b458-10284b6683b8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 12 00:16:30.781503 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: warnings.warn(msg) Oct 12 00:16:30.781503 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: Oct 12 00:16:30.782396 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e03b36-6a6c-4918-b458-10284b6683b8 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:16:30.794436 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 586/1173] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Oct 12 00:16:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:16:30.804455 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf774c4c-0cb4-48ab-8634-e845be1f22e1 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:16:30.813546 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf774c4c-0cb4-48ab-8634-e845be1f22e1 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:16:30.873928 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:16:30.907068 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a7ce15-8dff-4bfd-95af-1f82a1a6e6cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:16:30.907705 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a7ce15-8dff-4bfd-95af-1f82a1a6e6cb admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:16:30.907980 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a7ce15-8dff-4bfd-95af-1f82a1a6e6cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:16:30.908245 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a7ce15-8dff-4bfd-95af-1f82a1a6e6cb admin admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:16:30.908877 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a7ce15-8dff-4bfd-95af-1f82a1a6e6cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:16:30.909993 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: WARNING py.warnings [None req-f1a7ce15-8dff-4bfd-95af-1f82a1a6e6cb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 12 00:16:30.909993 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: warnings.warn(msg) Oct 12 00:16:30.909993 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: Oct 12 00:16:30.911174 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a7ce15-8dff-4bfd-95af-1f82a1a6e6cb admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:16:31.044962 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 588/1174] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:16:30 2022] POST /identity/v3/projects => generated 379 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:16:31.049232 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e03b36-6a6c-4918-b458-10284b6683b8 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:16:31.050780 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-43e03b36-6a6c-4918-b458-10284b6683b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:16:31.083586 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e03b36-6a6c-4918-b458-10284b6683b8 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:16:31.175111 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cdf9dd-2cc4-4092-86a7-45bdeac14fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:16:31.175676 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cdf9dd-2cc4-4092-86a7-45bdeac14fa8 admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:16:31.175944 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cdf9dd-2cc4-4092-86a7-45bdeac14fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:16:31.176207 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cdf9dd-2cc4-4092-86a7-45bdeac14fa8 admin admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:16:31.176820 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cdf9dd-2cc4-4092-86a7-45bdeac14fa8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:16:31.177812 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: WARNING py.warnings [None req-e2cdf9dd-2cc4-4092-86a7-45bdeac14fa8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 12 00:16:31.177812 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: warnings.warn(msg) Oct 12 00:16:31.177812 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: Oct 12 00:16:31.179033 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cdf9dd-2cc4-4092-86a7-45bdeac14fa8 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:16:31.301404 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 587/1175] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:16:31 2022] POST /identity/v3/users => generated 290 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:16:31.305560 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a7ce15-8dff-4bfd-95af-1f82a1a6e6cb admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:16:31.307121 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-f1a7ce15-8dff-4bfd-95af-1f82a1a6e6cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:16:31.367633 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a7ce15-8dff-4bfd-95af-1f82a1a6e6cb admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:16:31.449523 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:16:31.460617 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf0e40e-11d2-472c-89a6-dcea1250a8e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:16:31.464171 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf0e40e-11d2-472c-89a6-dcea1250a8e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:16:31.464171 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf0e40e-11d2-472c-89a6-dcea1250a8e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:16:31.464171 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf0e40e-11d2-472c-89a6-dcea1250a8e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:16:31.464171 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf0e40e-11d2-472c-89a6-dcea1250a8e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:16:31.464171 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf0e40e-11d2-472c-89a6-dcea1250a8e0 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:16:31.475826 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 589/1176] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:16:31 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:16:31.479901 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cdf9dd-2cc4-4092-86a7-45bdeac14fa8 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:16:31.481465 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-e2cdf9dd-2cc4-4092-86a7-45bdeac14fa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:16:31.492746 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cdf9dd-2cc4-4092-86a7-45bdeac14fa8 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:16:31.520730 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:16:31.593151 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ec0fb2-f723-4f4a-9688-3664b504f292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:16:31.593760 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ec0fb2-f723-4f4a-9688-3664b504f292 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:16:31.594039 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ec0fb2-f723-4f4a-9688-3664b504f292 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:16:31.594310 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ec0fb2-f723-4f4a-9688-3664b504f292 admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:16:31.594941 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ec0fb2-f723-4f4a-9688-3664b504f292 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:16:31.596560 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ec0fb2-f723-4f4a-9688-3664b504f292 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:16:31.609362 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 588/1177] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:16:31 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:16:31.613534 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf0e40e-11d2-472c-89a6-dcea1250a8e0 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:16:31.615075 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-bdf0e40e-11d2-472c-89a6-dcea1250a8e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:16:31.649665 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf0e40e-11d2-472c-89a6-dcea1250a8e0 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:16:31.743964 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12ec410-3562-45a1-9772-7c4a652b4d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:16:31.744551 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12ec410-3562-45a1-9772-7c4a652b4d9a admin admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:16:31.744827 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12ec410-3562-45a1-9772-7c4a652b4d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:16:31.745095 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12ec410-3562-45a1-9772-7c4a652b4d9a admin admin] PATH_INFO: `/v3/projects/d41a52cc39d0472da4a41bba359fdda0/users/16cbdeac9c6a48a8a2ec01bfb956f78b/roles/ef689f96442945e8a7bc8cbfee13d909` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:16:31.768442 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12ec410-3562-45a1-9772-7c4a652b4d9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d41a52cc39d0472da4a41bba359fdda0, user_id=16cbdeac9c6a48a8a2ec01bfb956f78b, role_id=ef689f96442945e8a7bc8cbfee13d909)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:16:31.771072 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12ec410-3562-45a1-9772-7c4a652b4d9a admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:16:31.870831 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 590/1178] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:16:31 2022] PUT /identity/v3/projects/d41a52cc39d0472da4a41bba359fdda0/users/16cbdeac9c6a48a8a2ec01bfb956f78b/roles/ef689f96442945e8a7bc8cbfee13d909 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:16:31.882748 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9f7064-a572-41eb-891b-399775219a78 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:16:31.883038 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9f7064-a572-41eb-891b-399775219a78 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:16:31.883310 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9f7064-a572-41eb-891b-399775219a78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:16:31.941062 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-ec9f7064-a572-41eb-891b-399775219a78 None None] MFA Rules not processed for user `16cbdeac9c6a48a8a2ec01bfb956f78b`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:16:31.942263 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-ec9f7064-a572-41eb-891b-399775219a78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:16:32.008747 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 589/1179] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:16:31 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:16:32.022042 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940896cd-5fbc-4bca-b099-248559c02f44 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:16:32.022329 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940896cd-5fbc-4bca-b099-248559c02f44 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:16:32.022602 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940896cd-5fbc-4bca-b099-248559c02f44 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:16:32.023986 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 591/1180] 10.222.0.36 () {54 vars in 963 bytes} [Wed Oct 12 00:16:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:16:32.028750 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d668d8-b823-4128-94c5-b9a26e879617 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:16:32.029034 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d668d8-b823-4128-94c5-b9a26e879617 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:16:32.029311 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d668d8-b823-4128-94c5-b9a26e879617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:16:32.089499 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:16:32.097649 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-a8d668d8-b823-4128-94c5-b9a26e879617 None None] MFA Rules not processed for user `7f87febbbaa94626807a98da4a771ab1`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:16:32.098845 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-a8d668d8-b823-4128-94c5-b9a26e879617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:16:32.173437 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 590/1181] 10.222.0.36 () {58 vars in 1060 bytes} [Wed Oct 12 00:16:32 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:16:32.181876 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940896cd-5fbc-4bca-b099-248559c02f44 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:16:32.184357 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-940896cd-5fbc-4bca-b099-248559c02f44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:16:32.235656 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:16:32.250585 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940896cd-5fbc-4bca-b099-248559c02f44 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:16:32.262872 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c154d5-3df9-4786-962f-a57d25e020ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:16:32.263450 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c154d5-3df9-4786-962f-a57d25e020ac service cinder] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:16:32.263723 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c154d5-3df9-4786-962f-a57d25e020ac service cinder] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:16:32.263988 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c154d5-3df9-4786-962f-a57d25e020ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:16:32.266104 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-d6c154d5-3df9-4786-962f-a57d25e020ac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:16:32.276951 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c154d5-3df9-4786-962f-a57d25e020ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:16:32.278382 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c154d5-3df9-4786-962f-a57d25e020ac service cinder] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:16:32.296346 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 592/1182] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:16:32 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:16:32.341612 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c5950-7582-46ff-a722-b22853590b99 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:16:32.341920 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c5950-7582-46ff-a722-b22853590b99 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:16:32.342188 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c5950-7582-46ff-a722-b22853590b99 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:16:32.343445 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 591/1183] 10.222.0.36 () {52 vars in 830 bytes} [Wed Oct 12 00:16:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:16:32.346876 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c154d5-3df9-4786-962f-a57d25e020ac service cinder] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:16:32.348340 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-d6c154d5-3df9-4786-962f-a57d25e020ac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:16:32.432560 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c154d5-3df9-4786-962f-a57d25e020ac service cinder] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:16:32.437159 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6103b22-5fcb-400c-9ec9-2f0e77be9e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:16:32.437781 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6103b22-5fcb-400c-9ec9-2f0e77be9e9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:16:32.438059 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6103b22-5fcb-400c-9ec9-2f0e77be9e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:16:32.438328 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6103b22-5fcb-400c-9ec9-2f0e77be9e9d admin admin] PATH_INFO: `/v3/users/16cbdeac9c6a48a8a2ec01bfb956f78b` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:16:32.439464 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6103b22-5fcb-400c-9ec9-2f0e77be9e9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=16cbdeac9c6a48a8a2ec01bfb956f78b)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:16:32.440479 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: WARNING py.warnings [None req-f6103b22-5fcb-400c-9ec9-2f0e77be9e9d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 12 00:16:32.440479 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: warnings.warn(msg) Oct 12 00:16:32.440479 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: Oct 12 00:16:32.441662 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6103b22-5fcb-400c-9ec9-2f0e77be9e9d admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:16:32.641521 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-f6103b22-5fcb-400c-9ec9-2f0e77be9e9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16cbdeac9c6a48a8a2ec01bfb956f78b'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:16:32.642248 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-f6103b22-5fcb-400c-9ec9-2f0e77be9e9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16cbdeac9c6a48a8a2ec01bfb956f78b'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:16:32.667751 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-f6103b22-5fcb-400c-9ec9-2f0e77be9e9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16cbdeac9c6a48a8a2ec01bfb956f78b'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:16:32.761956 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-f6103b22-5fcb-400c-9ec9-2f0e77be9e9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16cbdeac9c6a48a8a2ec01bfb956f78b'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:16:32.762673 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-f6103b22-5fcb-400c-9ec9-2f0e77be9e9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16cbdeac9c6a48a8a2ec01bfb956f78b'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:16:32.783557 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 593/1184] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:16:32 2022] DELETE /identity/v3/users/16cbdeac9c6a48a8a2ec01bfb956f78b => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:16:32.787972 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd28b71-7a16-41ed-8509-6f2a66c854e4 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:16:32.788442 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd28b71-7a16-41ed-8509-6f2a66c854e4 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:16:32.788581 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd28b71-7a16-41ed-8509-6f2a66c854e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:16:32.816084 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-acd28b71-7a16-41ed-8509-6f2a66c854e4 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:16:32.817223 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-acd28b71-7a16-41ed-8509-6f2a66c854e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:16:32.874030 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 592/1185] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:16:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:16:32.885104 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6103b22-5fcb-400c-9ec9-2f0e77be9e9d admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:16:32.886786 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-f6103b22-5fcb-400c-9ec9-2f0e77be9e9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:16:32.913914 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:16:32.966062 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6103b22-5fcb-400c-9ec9-2f0e77be9e9d admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:16:32.970420 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3533a24-a493-4d81-9871-677edeef1341 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:16:32.971052 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3533a24-a493-4d81-9871-677edeef1341 service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:16:32.971327 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3533a24-a493-4d81-9871-677edeef1341 service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:16:32.971622 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3533a24-a493-4d81-9871-677edeef1341 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:16:32.980299 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3533a24-a493-4d81-9871-677edeef1341 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:16:32.981725 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3533a24-a493-4d81-9871-677edeef1341 service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:16:32.991047 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 594/1186] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:16:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:16:33.496000 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd28b71-7a16-41ed-8509-6f2a66c854e4 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:16:33.497815 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-acd28b71-7a16-41ed-8509-6f2a66c854e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:16:33.509871 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd28b71-7a16-41ed-8509-6f2a66c854e4 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:16:33.514855 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27814082-3e02-49f7-a3d6-343aaef5ba70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:16:33.516331 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27814082-3e02-49f7-a3d6-343aaef5ba70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:16:33.517228 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27814082-3e02-49f7-a3d6-343aaef5ba70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:16:33.517509 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27814082-3e02-49f7-a3d6-343aaef5ba70 admin admin] PATH_INFO: `/v3/projects/d41a52cc39d0472da4a41bba359fdda0` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:16:33.518908 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27814082-3e02-49f7-a3d6-343aaef5ba70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d41a52cc39d0472da4a41bba359fdda0)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:16:33.519987 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: WARNING py.warnings [None req-27814082-3e02-49f7-a3d6-343aaef5ba70 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 12 00:16:33.519987 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: warnings.warn(msg) Oct 12 00:16:33.519987 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: Oct 12 00:16:33.521801 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27814082-3e02-49f7-a3d6-343aaef5ba70 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:16:33.642887 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-27814082-3e02-49f7-a3d6-343aaef5ba70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd41a52cc39d0472da4a41bba359fdda0', 'request_id': 'req-27814082-3e02-49f7-a3d6-343aaef5ba70'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:16:33.655979 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-27814082-3e02-49f7-a3d6-343aaef5ba70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd41a52cc39d0472da4a41bba359fdda0', 'request_id': 'req-27814082-3e02-49f7-a3d6-343aaef5ba70'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:16:33.723116 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-27814082-3e02-49f7-a3d6-343aaef5ba70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd41a52cc39d0472da4a41bba359fdda0', 'request_id': 'req-27814082-3e02-49f7-a3d6-343aaef5ba70'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:16:33.734858 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-27814082-3e02-49f7-a3d6-343aaef5ba70 admin admin] The token cache is being invalidate because project d41a52cc39d0472da4a41bba359fdda0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81747) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:16:33.735322 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-27814082-3e02-49f7-a3d6-343aaef5ba70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:16:33.735846 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-27814082-3e02-49f7-a3d6-343aaef5ba70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:16:33.737425 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 593/1187] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:16:33 2022] DELETE /identity/v3/projects/d41a52cc39d0472da4a41bba359fdda0 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:16:43.049320 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=25 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:16:43.731774 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:17:58.567792 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d66594-cf6d-476c-be8b-e3d991354dbf None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:17:58.568319 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d66594-cf6d-476c-be8b-e3d991354dbf None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:17:58.568319 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d66594-cf6d-476c-be8b-e3d991354dbf None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:17:58.569717 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 595/1188] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 12 00:17:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:17:58.574475 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce0ac5-5cf9-43f7-ad73-498cb7cd3914 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:17:58.574927 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce0ac5-5cf9-43f7-ad73-498cb7cd3914 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:17:58.575030 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce0ac5-5cf9-43f7-ad73-498cb7cd3914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:17:58.614611 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-74ce0ac5-5cf9-43f7-ad73-498cb7cd3914 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:17:58.615792 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-74ce0ac5-5cf9-43f7-ad73-498cb7cd3914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:17:58.673139 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 594/1189] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 12 00:17:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:17:58.824287 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d66594-cf6d-476c-be8b-e3d991354dbf None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:17:58.826029 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-b9d66594-cf6d-476c-be8b-e3d991354dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:17:58.927055 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d66594-cf6d-476c-be8b-e3d991354dbf None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:17:58.932554 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36321bff-b2b0-4ded-a401-7ea4fb1ee617 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a515b75aa3944912ad2364ffd7924f40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:17:58.933160 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36321bff-b2b0-4ded-a401-7ea4fb1ee617 service nova] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:17:58.933435 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36321bff-b2b0-4ded-a401-7ea4fb1ee617 service nova] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:17:58.933697 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36321bff-b2b0-4ded-a401-7ea4fb1ee617 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:17:58.942124 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36321bff-b2b0-4ded-a401-7ea4fb1ee617 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:17:58.943836 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36321bff-b2b0-4ded-a401-7ea4fb1ee617 service nova] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:17:58.953233 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 596/1190] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:17:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:18:00.025675 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a1eee0-2ff7-46b1-98be-82df09b5a7ba None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:18:00.026238 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a1eee0-2ff7-46b1-98be-82df09b5a7ba None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:18:00.026238 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a1eee0-2ff7-46b1-98be-82df09b5a7ba None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:18:00.027594 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 595/1191] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 12 00:18:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:18:00.033706 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4c0d9e-b73f-4a8e-b944-b53ed3080aa6 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:18:00.034162 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4c0d9e-b73f-4a8e-b944-b53ed3080aa6 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:18:00.034265 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4c0d9e-b73f-4a8e-b944-b53ed3080aa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:18:00.054539 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-ce4c0d9e-b73f-4a8e-b944-b53ed3080aa6 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:18:00.055780 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-ce4c0d9e-b73f-4a8e-b944-b53ed3080aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:18:00.063591 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 597/1192] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 12 00:18:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:18:00.214431 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a1eee0-2ff7-46b1-98be-82df09b5a7ba None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:18:00.229441 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a1eee0-2ff7-46b1-98be-82df09b5a7ba None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:18:00.234850 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcf98c2-e2b1-4f2d-b06b-f50d07492c73 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a515b75aa3944912ad2364ffd7924f40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:18:00.235473 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf98c2-e2b1-4f2d-b06b-f50d07492c73 service nova] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:18:00.235753 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf98c2-e2b1-4f2d-b06b-f50d07492c73 service nova] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:18:00.236021 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf98c2-e2b1-4f2d-b06b-f50d07492c73 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:18:00.243887 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcf98c2-e2b1-4f2d-b06b-f50d07492c73 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:18:00.245484 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcf98c2-e2b1-4f2d-b06b-f50d07492c73 service nova] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:18:00.255015 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 596/1193] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:18:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:18:00.291933 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae922da-29d2-472d-9341-6e4c68540289 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:18:00.292221 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae922da-29d2-472d-9341-6e4c68540289 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:18:00.292495 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae922da-29d2-472d-9341-6e4c68540289 None None] PATH_INFO: `/v3` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:18:00.293803 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 598/1194] 10.222.0.40 () {54 vars in 925 bytes} [Wed Oct 12 00:18:00 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:18:00.302912 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a057dd47-b0d5-47f7-9312-ce11353dc6c7 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:18:00.303194 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a057dd47-b0d5-47f7-9312-ce11353dc6c7 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:18:00.303497 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a057dd47-b0d5-47f7-9312-ce11353dc6c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:18:00.361870 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-a057dd47-b0d5-47f7-9312-ce11353dc6c7 None None] MFA Rules not processed for user `1e528da752ed4c78bc747215909f6569`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:18:00.363138 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-a057dd47-b0d5-47f7-9312-ce11353dc6c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:18:00.436981 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 597/1195] 10.222.0.40 () {58 vars in 1015 bytes} [Wed Oct 12 00:18:00 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:18:10.048694 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=49 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:18:10.434104 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=84 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:18:10.568514 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc84b73-6b55-469f-a8c7-b0b2851608be None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:18:10.568796 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc84b73-6b55-469f-a8c7-b0b2851608be None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:18:10.569067 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc84b73-6b55-469f-a8c7-b0b2851608be None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:18:10.570388 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 599/1196] 10.222.0.36 () {54 vars in 961 bytes} [Wed Oct 12 00:18:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:18:10.575487 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f26c05f-f8c8-44f8-80aa-2f19732d0d41 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:18:10.575772 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f26c05f-f8c8-44f8-80aa-2f19732d0d41 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:18:10.576045 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f26c05f-f8c8-44f8-80aa-2f19732d0d41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:18:10.596408 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-7f26c05f-f8c8-44f8-80aa-2f19732d0d41 None None] MFA Rules not processed for user `1e528da752ed4c78bc747215909f6569`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:18:10.598110 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-7f26c05f-f8c8-44f8-80aa-2f19732d0d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:18:10.606619 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 598/1197] 10.222.0.36 () {58 vars in 1058 bytes} [Wed Oct 12 00:18:10 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:18:10.614803 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc84b73-6b55-469f-a8c7-b0b2851608be None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:18:10.627963 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc84b73-6b55-469f-a8c7-b0b2851608be None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:18:10.632980 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23c9a3d-87c6-423c-a6e9-80e4d3bac44e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e528da752ed4c78bc747215909f6569', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:18:10.633574 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23c9a3d-87c6-423c-a6e9-80e4d3bac44e service placement] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:18:10.633851 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23c9a3d-87c6-423c-a6e9-80e4d3bac44e service placement] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:18:10.634121 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23c9a3d-87c6-423c-a6e9-80e4d3bac44e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:18:10.642619 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23c9a3d-87c6-423c-a6e9-80e4d3bac44e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:18:10.644289 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23c9a3d-87c6-423c-a6e9-80e4d3bac44e service placement] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:18:10.654722 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 600/1198] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:18:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:18:11.734348 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9253133-b3af-4cde-a632-192b71d83c12 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:18:11.734934 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9253133-b3af-4cde-a632-192b71d83c12 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:18:11.734934 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9253133-b3af-4cde-a632-192b71d83c12 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:18:11.736304 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 599/1199] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 12 00:18:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:18:11.742914 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3807e50-cf47-4e2c-97a5-55b44f93f097 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:18:11.743321 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3807e50-cf47-4e2c-97a5-55b44f93f097 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:18:11.743502 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3807e50-cf47-4e2c-97a5-55b44f93f097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:18:11.762470 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-d3807e50-cf47-4e2c-97a5-55b44f93f097 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:18:11.763702 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-d3807e50-cf47-4e2c-97a5-55b44f93f097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:18:11.771860 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 601/1200] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 12 00:18:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:18:11.925172 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9253133-b3af-4cde-a632-192b71d83c12 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:18:11.940179 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9253133-b3af-4cde-a632-192b71d83c12 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:18:11.945588 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bd48aa-0cca-44b9-9cb5-c74ad15d4089 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a515b75aa3944912ad2364ffd7924f40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:18:11.946185 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bd48aa-0cca-44b9-9cb5-c74ad15d4089 service nova] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:18:11.946461 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bd48aa-0cca-44b9-9cb5-c74ad15d4089 service nova] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:18:11.946723 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bd48aa-0cca-44b9-9cb5-c74ad15d4089 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:18:11.954706 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bd48aa-0cca-44b9-9cb5-c74ad15d4089 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:18:11.956583 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bd48aa-0cca-44b9-9cb5-c74ad15d4089 service nova] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:18:11.965716 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 600/1201] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:18:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:18:13.024001 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f74729a-c469-4816-a747-93ca5935c7c3 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:18:13.024452 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f74729a-c469-4816-a747-93ca5935c7c3 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:18:13.024555 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f74729a-c469-4816-a747-93ca5935c7c3 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:18:13.025888 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 602/1202] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 12 00:18:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:18:13.032120 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7446b2f-103e-4a5b-8258-d3f509c9a29e None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:18:13.032579 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7446b2f-103e-4a5b-8258-d3f509c9a29e None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:18:13.032685 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7446b2f-103e-4a5b-8258-d3f509c9a29e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:18:13.052497 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-e7446b2f-103e-4a5b-8258-d3f509c9a29e None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:18:13.053699 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-e7446b2f-103e-4a5b-8258-d3f509c9a29e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:18:13.061770 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 601/1203] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 12 00:18:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:18:13.211419 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f74729a-c469-4816-a747-93ca5935c7c3 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:18:13.225936 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f74729a-c469-4816-a747-93ca5935c7c3 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:18:13.231447 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dc3aa0-512f-4b48-9052-c78e2233c74c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a515b75aa3944912ad2364ffd7924f40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:18:13.231987 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dc3aa0-512f-4b48-9052-c78e2233c74c service nova] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:18:13.232265 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dc3aa0-512f-4b48-9052-c78e2233c74c service nova] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:18:13.232536 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dc3aa0-512f-4b48-9052-c78e2233c74c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:18:13.241045 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dc3aa0-512f-4b48-9052-c78e2233c74c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:18:13.242613 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dc3aa0-512f-4b48-9052-c78e2233c74c service nova] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:18:13.251763 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 603/1204] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:18:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:18:17.136536 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011584a2-c856-4b4a-9ccc-15d78d4b2033 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:18:17.137137 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011584a2-c856-4b4a-9ccc-15d78d4b2033 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:18:17.137137 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011584a2-c856-4b4a-9ccc-15d78d4b2033 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:18:17.138405 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 602/1205] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 12 00:18:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:18:17.143827 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca87d788-f02e-46c2-b94e-c1ec3aea6ca0 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:18:17.144825 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca87d788-f02e-46c2-b94e-c1ec3aea6ca0 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:18:17.144971 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca87d788-f02e-46c2-b94e-c1ec3aea6ca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:18:17.164115 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-ca87d788-f02e-46c2-b94e-c1ec3aea6ca0 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:18:17.165299 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-ca87d788-f02e-46c2-b94e-c1ec3aea6ca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:18:17.172951 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 604/1206] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 12 00:18:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:18:17.468375 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011584a2-c856-4b4a-9ccc-15d78d4b2033 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:18:17.470061 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-011584a2-c856-4b4a-9ccc-15d78d4b2033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:18:17.526644 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011584a2-c856-4b4a-9ccc-15d78d4b2033 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:18:17.530215 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dbea16-cb25-4d74-94a7-244ebc1b658c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:18:17.530803 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dbea16-cb25-4d74-94a7-244ebc1b658c service neutron] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:18:17.531076 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dbea16-cb25-4d74-94a7-244ebc1b658c service neutron] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:18:17.531341 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dbea16-cb25-4d74-94a7-244ebc1b658c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:18:17.541151 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dbea16-cb25-4d74-94a7-244ebc1b658c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:18:17.542574 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dbea16-cb25-4d74-94a7-244ebc1b658c service neutron] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:18:17.552025 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 603/1207] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:18:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:18:19.696309 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191529ec-d202-45fa-ac75-83bc9bc1d084 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:18:19.696874 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191529ec-d202-45fa-ac75-83bc9bc1d084 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:18:19.696874 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191529ec-d202-45fa-ac75-83bc9bc1d084 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:18:19.698172 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 605/1208] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 12 00:18:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:18:19.703313 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc8c433-d677-4efc-8362-73e64541ad93 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:18:19.703764 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc8c433-d677-4efc-8362-73e64541ad93 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:18:19.703884 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc8c433-d677-4efc-8362-73e64541ad93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:18:19.722993 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-2cc8c433-d677-4efc-8362-73e64541ad93 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:18:19.724211 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-2cc8c433-d677-4efc-8362-73e64541ad93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:18:19.731860 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 604/1209] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 12 00:18:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:18:20.017313 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191529ec-d202-45fa-ac75-83bc9bc1d084 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:18:20.019032 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-191529ec-d202-45fa-ac75-83bc9bc1d084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:18:20.031698 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191529ec-d202-45fa-ac75-83bc9bc1d084 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:18:20.035653 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3961651-22d4-4b2d-b12a-ce2564e27d8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:18:20.036236 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3961651-22d4-4b2d-b12a-ce2564e27d8d service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:18:20.036510 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3961651-22d4-4b2d-b12a-ce2564e27d8d service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:18:20.036775 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3961651-22d4-4b2d-b12a-ce2564e27d8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:18:20.045286 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3961651-22d4-4b2d-b12a-ce2564e27d8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:18:20.046687 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3961651-22d4-4b2d-b12a-ce2564e27d8d service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:18:20.056370 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 606/1210] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:18:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:18:23.723278 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bff00a-2249-4390-b680-dda8758a3702 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:18:23.723814 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bff00a-2249-4390-b680-dda8758a3702 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:18:23.723814 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bff00a-2249-4390-b680-dda8758a3702 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:18:23.725168 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 605/1211] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 12 00:18:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:18:23.730610 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bde88c-602e-45b2-9509-c17d18e43b5c None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:18:23.731033 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bde88c-602e-45b2-9509-c17d18e43b5c None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:18:23.731136 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bde88c-602e-45b2-9509-c17d18e43b5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:18:23.750767 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-75bde88c-602e-45b2-9509-c17d18e43b5c None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:18:23.753799 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-75bde88c-602e-45b2-9509-c17d18e43b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:18:23.760811 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 607/1212] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 12 00:18:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:18:24.065138 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bff00a-2249-4390-b680-dda8758a3702 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:18:24.067414 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-d2bff00a-2249-4390-b680-dda8758a3702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:18:24.124042 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bff00a-2249-4390-b680-dda8758a3702 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:18:24.127519 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4fe948-4dbb-484a-8729-a21f2e26db6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:18:24.128698 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4fe948-4dbb-484a-8729-a21f2e26db6c service cinder] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:18:24.128970 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4fe948-4dbb-484a-8729-a21f2e26db6c service cinder] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:18:24.129232 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4fe948-4dbb-484a-8729-a21f2e26db6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:18:24.137839 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4fe948-4dbb-484a-8729-a21f2e26db6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:18:24.139246 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4fe948-4dbb-484a-8729-a21f2e26db6c service cinder] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:18:24.148686 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 606/1213] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:18:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:18:26.279961 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4bd24a-fd24-4733-8a50-92c9ec29c72c None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:18:26.280583 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4bd24a-fd24-4733-8a50-92c9ec29c72c None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:18:26.280583 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4bd24a-fd24-4733-8a50-92c9ec29c72c None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:18:26.281851 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 608/1214] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 12 00:18:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:18:26.287575 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dc67f1-ac61-4564-a064-83a064486755 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:18:26.288043 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dc67f1-ac61-4564-a064-83a064486755 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:18:26.288167 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dc67f1-ac61-4564-a064-83a064486755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:18:26.308004 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-e5dc67f1-ac61-4564-a064-83a064486755 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:18:26.309215 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-e5dc67f1-ac61-4564-a064-83a064486755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:18:26.317124 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 607/1215] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 12 00:18:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:18:26.618915 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4bd24a-fd24-4733-8a50-92c9ec29c72c None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:18:26.620679 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-1f4bd24a-fd24-4733-8a50-92c9ec29c72c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:18:26.633497 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4bd24a-fd24-4733-8a50-92c9ec29c72c None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:18:26.637017 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7698d3f6-d02c-486e-9e87-0a8549f179a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:18:26.637616 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7698d3f6-d02c-486e-9e87-0a8549f179a2 service cinder] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:18:26.637893 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7698d3f6-d02c-486e-9e87-0a8549f179a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:18:26.638166 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7698d3f6-d02c-486e-9e87-0a8549f179a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:18:26.647841 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7698d3f6-d02c-486e-9e87-0a8549f179a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:18:26.649438 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7698d3f6-d02c-486e-9e87-0a8549f179a2 service cinder] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:18:26.659973 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 609/1216] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:18:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:18:36.303214 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=70 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:18:36.654941 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=33 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:19:50.209278 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3afa55b-9b18-4b71-a95d-6bf6695cb21e None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:50.210486 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3afa55b-9b18-4b71-a95d-6bf6695cb21e None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:50.210999 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3afa55b-9b18-4b71-a95d-6bf6695cb21e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:50.214949 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77823208-1117-4cd0-b913-992736c7549b None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:50.215908 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77823208-1117-4cd0-b913-992736c7549b None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:50.216440 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77823208-1117-4cd0-b913-992736c7549b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:50.262399 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-a3afa55b-9b18-4b71-a95d-6bf6695cb21e None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:19:50.263107 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-77823208-1117-4cd0-b913-992736c7549b None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:19:50.265672 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-a3afa55b-9b18-4b71-a95d-6bf6695cb21e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:19:50.266625 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-77823208-1117-4cd0-b913-992736c7549b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:19:50.288594 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 608/1217] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:19:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:19:50.289299 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 610/1218] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:19:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:19:50.307002 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67680f1-b0a0-4a71-83b0-b1fbf9c6a8bd None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:50.308777 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6b134b-d5b0-4057-9638-dad02119b868 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:50.309313 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6b134b-d5b0-4057-9638-dad02119b868 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:50.309822 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67680f1-b0a0-4a71-83b0-b1fbf9c6a8bd None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:50.310618 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67680f1-b0a0-4a71-83b0-b1fbf9c6a8bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:50.310922 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6b134b-d5b0-4057-9638-dad02119b868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:50.350672 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-db6b134b-d5b0-4057-9638-dad02119b868 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:19:50.351225 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-f67680f1-b0a0-4a71-83b0-b1fbf9c6a8bd None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:19:50.351918 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-db6b134b-d5b0-4057-9638-dad02119b868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:19:50.352933 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-f67680f1-b0a0-4a71-83b0-b1fbf9c6a8bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:19:50.367055 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 609/1219] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:19:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 00:19:50.369142 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa1a811-afe6-4623-bb4c-9648f4a57aec None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:50.369672 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa1a811-afe6-4623-bb4c-9648f4a57aec None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:50.370193 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa1a811-afe6-4623-bb4c-9648f4a57aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:50.372624 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 611/1220] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:19:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:19:50.373451 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6b134b-d5b0-4057-9638-dad02119b868 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:50.388522 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6b134b-d5b0-4057-9638-dad02119b868 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:50.401247 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cee51d0-b561-49e1-a4aa-23d036384215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:50.402025 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cee51d0-b561-49e1-a4aa-23d036384215 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:50.402313 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cee51d0-b561-49e1-a4aa-23d036384215 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:50.402586 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cee51d0-b561-49e1-a4aa-23d036384215 admin admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:50.403242 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cee51d0-b561-49e1-a4aa-23d036384215 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:19:50.403569 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cee51d0-b561-49e1-a4aa-23d036384215 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:50.407388 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: WARNING py.warnings [None req-4cee51d0-b561-49e1-a4aa-23d036384215 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 12 00:19:50.407388 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: warnings.warn(msg) Oct 12 00:19:50.407388 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: Oct 12 00:19:50.407388 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cee51d0-b561-49e1-a4aa-23d036384215 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:50.409251 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-7fa1a811-afe6-4623-bb4c-9648f4a57aec None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:19:50.410831 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-7fa1a811-afe6-4623-bb4c-9648f4a57aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:19:50.426175 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 612/1221] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Oct 12 00:19:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:19:50.427028 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cee51d0-b561-49e1-a4aa-23d036384215 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:50.428311 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 610/1222] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:19:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:19:50.434818 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa1a811-afe6-4623-bb4c-9648f4a57aec None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:50.452668 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa1a811-afe6-4623-bb4c-9648f4a57aec None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:50.454453 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cee51d0-b561-49e1-a4aa-23d036384215 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:50.460151 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40350d40-d2d1-4ad7-8ee0-cafc27588817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:50.461078 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40350d40-d2d1-4ad7-8ee0-cafc27588817 admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:50.461651 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40350d40-d2d1-4ad7-8ee0-cafc27588817 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:50.462256 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40350d40-d2d1-4ad7-8ee0-cafc27588817 admin admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:50.463135 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40350d40-d2d1-4ad7-8ee0-cafc27588817 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:50.464399 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: WARNING py.warnings [None req-40350d40-d2d1-4ad7-8ee0-cafc27588817 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 12 00:19:50.464399 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: warnings.warn(msg) Oct 12 00:19:50.464399 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: Oct 12 00:19:50.466072 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40350d40-d2d1-4ad7-8ee0-cafc27588817 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:50.470174 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b023373-e4de-4722-943b-0cf1a46f3632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:50.470806 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b023373-e4de-4722-943b-0cf1a46f3632 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:50.471319 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b023373-e4de-4722-943b-0cf1a46f3632 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:50.471653 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b023373-e4de-4722-943b-0cf1a46f3632 admin admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:50.472308 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b023373-e4de-4722-943b-0cf1a46f3632 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:19:50.472611 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b023373-e4de-4722-943b-0cf1a46f3632 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:50.474215 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b023373-e4de-4722-943b-0cf1a46f3632 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:50.489888 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 613/1223] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Oct 12 00:19:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:19:50.491159 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99457aaf-2348-4362-8eb6-c4ec5bac1a4f None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:50.491751 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99457aaf-2348-4362-8eb6-c4ec5bac1a4f None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:50.492113 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99457aaf-2348-4362-8eb6-c4ec5bac1a4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:50.512954 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-99457aaf-2348-4362-8eb6-c4ec5bac1a4f None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:19:50.523297 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-99457aaf-2348-4362-8eb6-c4ec5bac1a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:19:50.523447 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 614/1224] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:19:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:19:50.524731 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5b931b-0366-450d-ba8e-aecc1283cb7a None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:50.525011 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5b931b-0366-450d-ba8e-aecc1283cb7a None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:50.525283 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5b931b-0366-450d-ba8e-aecc1283cb7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:50.544871 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-eb5b931b-0366-450d-ba8e-aecc1283cb7a None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:19:50.546073 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-eb5b931b-0366-450d-ba8e-aecc1283cb7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:19:50.554086 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 615/1225] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:19:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:19:50.555233 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b7f04e-0c54-4d3b-acea-ec9f71428cf6 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:50.555542 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b7f04e-0c54-4d3b-acea-ec9f71428cf6 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:50.555815 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b7f04e-0c54-4d3b-acea-ec9f71428cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:50.575415 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-f1b7f04e-0c54-4d3b-acea-ec9f71428cf6 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:19:50.576615 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-f1b7f04e-0c54-4d3b-acea-ec9f71428cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:19:50.585468 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 616/1226] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:19:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:19:50.586629 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9531f6-285e-44e7-8688-a57b4234dd06 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:50.586910 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9531f6-285e-44e7-8688-a57b4234dd06 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:50.587182 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9531f6-285e-44e7-8688-a57b4234dd06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:50.606884 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-cc9531f6-285e-44e7-8688-a57b4234dd06 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:19:50.608103 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-cc9531f6-285e-44e7-8688-a57b4234dd06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:19:50.615938 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 617/1227] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:19:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:19:50.617157 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5537574d-6652-41a2-bf87-c39026c1082d None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:50.617391 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5537574d-6652-41a2-bf87-c39026c1082d None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:50.617666 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5537574d-6652-41a2-bf87-c39026c1082d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:50.626612 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 611/1228] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:19:50 2022] POST /identity/v3/projects => generated 363 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 00:19:50.633853 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40350d40-d2d1-4ad7-8ee0-cafc27588817 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:50.637592 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-5537574d-6652-41a2-bf87-c39026c1082d None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:19:50.638807 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-5537574d-6652-41a2-bf87-c39026c1082d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:19:50.691093 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40350d40-d2d1-4ad7-8ee0-cafc27588817 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:50.695990 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75f477c-74bf-4e57-a644-5fe11755d4c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:50.696598 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75f477c-74bf-4e57-a644-5fe11755d4c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:50.696899 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75f477c-74bf-4e57-a644-5fe11755d4c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:50.697172 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75f477c-74bf-4e57-a644-5fe11755d4c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:50.697828 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75f477c-74bf-4e57-a644-5fe11755d4c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:50.699557 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 618/1229] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:19:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:19:50.699772 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75f477c-74bf-4e57-a644-5fe11755d4c4 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:50.700793 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2725b40-90b9-4b0e-909f-8e410f50f622 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:50.701079 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2725b40-90b9-4b0e-909f-8e410f50f622 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:50.701353 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2725b40-90b9-4b0e-909f-8e410f50f622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:50.721172 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-a2725b40-90b9-4b0e-909f-8e410f50f622 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:19:50.722300 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-a2725b40-90b9-4b0e-909f-8e410f50f622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:19:50.730297 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 619/1230] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:19:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:19:50.731192 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2725b40-90b9-4b0e-909f-8e410f50f622 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:50.740394 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2725b40-90b9-4b0e-909f-8e410f50f622 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:50.745074 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb779454-8f28-4520-9623-6f470ffae7ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:50.745665 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb779454-8f28-4520-9623-6f470ffae7ed admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:50.745943 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb779454-8f28-4520-9623-6f470ffae7ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:50.746244 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb779454-8f28-4520-9623-6f470ffae7ed admin admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:50.746900 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb779454-8f28-4520-9623-6f470ffae7ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:19:50.747196 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb779454-8f28-4520-9623-6f470ffae7ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:50.748839 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb779454-8f28-4520-9623-6f470ffae7ed admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:50.760745 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 620/1231] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Oct 12 00:19:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:19:50.761856 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e2e27-3c30-48d5-9ea1-4ec6122d0021 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:50.762138 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e2e27-3c30-48d5-9ea1-4ec6122d0021 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:50.762409 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e2e27-3c30-48d5-9ea1-4ec6122d0021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:50.781253 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 612/1232] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:19:50 2022] POST /identity/v3/projects => generated 383 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:19:50.781253 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27077783-ca15-4cc3-a328-429f6bdc8d78 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:50.781594 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27077783-ca15-4cc3-a328-429f6bdc8d78 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:50.785111 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27077783-ca15-4cc3-a328-429f6bdc8d78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:50.785241 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-bc5e2e27-3c30-48d5-9ea1-4ec6122d0021 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:19:50.785241 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-bc5e2e27-3c30-48d5-9ea1-4ec6122d0021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:19:50.801717 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-27077783-ca15-4cc3-a328-429f6bdc8d78 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:19:50.802937 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-27077783-ca15-4cc3-a328-429f6bdc8d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:19:50.843883 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 621/1233] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:19:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:19:50.845130 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d114c9-7da5-42bb-99b4-870edcfca401 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:50.845415 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d114c9-7da5-42bb-99b4-870edcfca401 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:50.845687 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d114c9-7da5-42bb-99b4-870edcfca401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:50.864568 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-80d114c9-7da5-42bb-99b4-870edcfca401 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:19:50.865773 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-80d114c9-7da5-42bb-99b4-870edcfca401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:19:50.866896 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 613/1234] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:19:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:19:50.867806 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27077783-ca15-4cc3-a328-429f6bdc8d78 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:50.875609 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 622/1235] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:19:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:19:50.876929 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064f0d49-f4b9-4a5f-bf1c-38c6679c229e None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:50.877215 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064f0d49-f4b9-4a5f-bf1c-38c6679c229e None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:50.877487 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064f0d49-f4b9-4a5f-bf1c-38c6679c229e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:50.881128 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27077783-ca15-4cc3-a328-429f6bdc8d78 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:50.885772 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200e0a03-7890-4b59-b546-279e80a4e958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:50.886375 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200e0a03-7890-4b59-b546-279e80a4e958 admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:50.886657 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200e0a03-7890-4b59-b546-279e80a4e958 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:50.886931 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200e0a03-7890-4b59-b546-279e80a4e958 admin admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:50.887589 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200e0a03-7890-4b59-b546-279e80a4e958 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:50.889771 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200e0a03-7890-4b59-b546-279e80a4e958 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:50.897527 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-064f0d49-f4b9-4a5f-bf1c-38c6679c229e None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:19:50.898923 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-064f0d49-f4b9-4a5f-bf1c-38c6679c229e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:19:50.908600 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 623/1236] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:19:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:19:50.912554 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4204899-342d-4524-99da-926ace2e150c None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:50.912554 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4204899-342d-4524-99da-926ace2e150c None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:50.912554 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4204899-342d-4524-99da-926ace2e150c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:50.917481 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:19:50.930233 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-e4204899-342d-4524-99da-926ace2e150c None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:19:50.931548 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-e4204899-342d-4524-99da-926ace2e150c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:19:50.939953 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 624/1237] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:19:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:19:50.940757 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4204899-342d-4524-99da-926ace2e150c None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:50.950330 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4204899-342d-4524-99da-926ace2e150c None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:50.955557 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9601d3-9ee8-4921-8eaa-7198c76e76fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:50.956123 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9601d3-9ee8-4921-8eaa-7198c76e76fb admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:50.956401 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9601d3-9ee8-4921-8eaa-7198c76e76fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:50.956666 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9601d3-9ee8-4921-8eaa-7198c76e76fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:50.957270 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9601d3-9ee8-4921-8eaa-7198c76e76fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:50.959163 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9601d3-9ee8-4921-8eaa-7198c76e76fb admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:50.979547 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 614/1238] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:19:50 2022] POST /identity/v3/users => generated 282 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:19:50.980561 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200e0a03-7890-4b59-b546-279e80a4e958 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:50.991279 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200e0a03-7890-4b59-b546-279e80a4e958 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:50.996132 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e255c311-5478-4c90-ad54-30a244c92e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:50.996721 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e255c311-5478-4c90-ad54-30a244c92e4f admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:50.997001 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e255c311-5478-4c90-ad54-30a244c92e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:50.997279 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e255c311-5478-4c90-ad54-30a244c92e4f admin admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:50.997899 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e255c311-5478-4c90-ad54-30a244c92e4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:51.000463 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e255c311-5478-4c90-ad54-30a244c92e4f admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:51.062890 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 625/1239] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:19:50 2022] POST /identity/v3/projects => generated 373 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:19:51.063827 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9601d3-9ee8-4921-8eaa-7198c76e76fb admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:51.088146 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:19:51.094272 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 615/1240] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:19:50 2022] POST /identity/v3/users => generated 292 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:19:51.095060 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e255c311-5478-4c90-ad54-30a244c92e4f admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:51.104512 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e255c311-5478-4c90-ad54-30a244c92e4f admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:51.123754 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9601d3-9ee8-4921-8eaa-7198c76e76fb admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:51.128946 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347d728e-9e38-4c52-ad3b-60f598b46337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:51.129552 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347d728e-9e38-4c52-ad3b-60f598b46337 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:51.129836 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347d728e-9e38-4c52-ad3b-60f598b46337 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:51.130107 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347d728e-9e38-4c52-ad3b-60f598b46337 admin admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:51.130745 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347d728e-9e38-4c52-ad3b-60f598b46337 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:19:51.131044 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347d728e-9e38-4c52-ad3b-60f598b46337 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:51.132667 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347d728e-9e38-4c52-ad3b-60f598b46337 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:51.144214 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 626/1241] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Oct 12 00:19:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:19:51.144773 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347d728e-9e38-4c52-ad3b-60f598b46337 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:51.154961 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347d728e-9e38-4c52-ad3b-60f598b46337 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:51.159510 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e6abd6-f4f1-4094-8e04-53c259651183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:51.170706 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e6abd6-f4f1-4094-8e04-53c259651183 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:51.170706 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e6abd6-f4f1-4094-8e04-53c259651183 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:51.170706 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e6abd6-f4f1-4094-8e04-53c259651183 admin admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:51.170706 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e6abd6-f4f1-4094-8e04-53c259651183 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:19:51.170706 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e6abd6-f4f1-4094-8e04-53c259651183 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:51.171316 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc42e7b-b9e9-4bfe-b9fb-21ba9e324414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:51.171442 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e6abd6-f4f1-4094-8e04-53c259651183 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:51.172013 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc42e7b-b9e9-4bfe-b9fb-21ba9e324414 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:51.172013 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc42e7b-b9e9-4bfe-b9fb-21ba9e324414 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:51.172013 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc42e7b-b9e9-4bfe-b9fb-21ba9e324414 admin admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:51.172013 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc42e7b-b9e9-4bfe-b9fb-21ba9e324414 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:19:51.172013 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc42e7b-b9e9-4bfe-b9fb-21ba9e324414 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:51.172013 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc42e7b-b9e9-4bfe-b9fb-21ba9e324414 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:51.175827 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 627/1242] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Oct 12 00:19:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:19:51.176461 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e6abd6-f4f1-4094-8e04-53c259651183 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:51.178663 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 616/1243] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Oct 12 00:19:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:19:51.179542 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc42e7b-b9e9-4bfe-b9fb-21ba9e324414 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:51.187014 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e6abd6-f4f1-4094-8e04-53c259651183 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:51.194071 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dc6329-b1da-4637-91d0-5ee5decb5ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:51.194674 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dc6329-b1da-4637-91d0-5ee5decb5ef3 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:51.194948 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dc6329-b1da-4637-91d0-5ee5decb5ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:51.195212 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dc6329-b1da-4637-91d0-5ee5decb5ef3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:51.195873 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dc6329-b1da-4637-91d0-5ee5decb5ef3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:19:51.196170 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dc6329-b1da-4637-91d0-5ee5decb5ef3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:51.197038 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc42e7b-b9e9-4bfe-b9fb-21ba9e324414 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:51.197785 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dc6329-b1da-4637-91d0-5ee5decb5ef3 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:51.205191 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ac124a-67b2-45c7-811a-c07849387fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:51.205813 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ac124a-67b2-45c7-811a-c07849387fe5 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:51.206091 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ac124a-67b2-45c7-811a-c07849387fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:51.206630 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ac124a-67b2-45c7-811a-c07849387fe5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:51.206998 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ac124a-67b2-45c7-811a-c07849387fe5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:19:51.207301 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ac124a-67b2-45c7-811a-c07849387fe5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:51.208924 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ac124a-67b2-45c7-811a-c07849387fe5 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:51.209755 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 628/1244] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Oct 12 00:19:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:19:51.210634 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dc6329-b1da-4637-91d0-5ee5decb5ef3 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:51.220528 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dc6329-b1da-4637-91d0-5ee5decb5ef3 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:51.225277 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44deb72d-cc36-4bdf-9e06-2dec2b533ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:51.225862 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44deb72d-cc36-4bdf-9e06-2dec2b533ccc admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:51.226137 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44deb72d-cc36-4bdf-9e06-2dec2b533ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:51.226403 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44deb72d-cc36-4bdf-9e06-2dec2b533ccc admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:51.227033 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44deb72d-cc36-4bdf-9e06-2dec2b533ccc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:51.227658 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 617/1245] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Oct 12 00:19:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:19:51.228454 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ac124a-67b2-45c7-811a-c07849387fe5 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:51.228641 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44deb72d-cc36-4bdf-9e06-2dec2b533ccc admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:51.245967 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ac124a-67b2-45c7-811a-c07849387fe5 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:51.246150 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 629/1246] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:19:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:19:51.246150 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44deb72d-cc36-4bdf-9e06-2dec2b533ccc admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:51.246331 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9d80e1-60e9-463b-8924-70d8f8b77031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:51.246331 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d80e1-60e9-463b-8924-70d8f8b77031 admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:51.246331 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d80e1-60e9-463b-8924-70d8f8b77031 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:51.246331 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d80e1-60e9-463b-8924-70d8f8b77031 admin admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:51.246697 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9d80e1-60e9-463b-8924-70d8f8b77031 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:51.247757 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9d80e1-60e9-463b-8924-70d8f8b77031 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:51.252366 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44deb72d-cc36-4bdf-9e06-2dec2b533ccc admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:51.256923 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755a2c8f-5a30-465a-9f6b-df32c95ed2cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:51.257507 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a2c8f-5a30-465a-9f6b-df32c95ed2cd admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:51.257781 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a2c8f-5a30-465a-9f6b-df32c95ed2cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:51.258048 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a2c8f-5a30-465a-9f6b-df32c95ed2cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:51.258710 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755a2c8f-5a30-465a-9f6b-df32c95ed2cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:51.260298 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755a2c8f-5a30-465a-9f6b-df32c95ed2cd admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:51.270392 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 630/1247] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:19:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:19:51.272922 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755a2c8f-5a30-465a-9f6b-df32c95ed2cd admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:51.280099 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755a2c8f-5a30-465a-9f6b-df32c95ed2cd admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:51.284557 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef8a537-db0c-4db5-8e23-49ea91877d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:51.285099 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef8a537-db0c-4db5-8e23-49ea91877d2a admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:51.285375 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef8a537-db0c-4db5-8e23-49ea91877d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:51.285645 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef8a537-db0c-4db5-8e23-49ea91877d2a admin admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:51.286254 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef8a537-db0c-4db5-8e23-49ea91877d2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:51.288111 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef8a537-db0c-4db5-8e23-49ea91877d2a admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:51.324537 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 618/1248] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:19:51 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:19:51.324537 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9d80e1-60e9-463b-8924-70d8f8b77031 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:51.331749 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9d80e1-60e9-463b-8924-70d8f8b77031 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:51.336490 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d65cff-0b94-4fc0-b44f-5476a3a09f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:51.337069 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d65cff-0b94-4fc0-b44f-5476a3a09f8f admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:51.337371 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d65cff-0b94-4fc0-b44f-5476a3a09f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:51.337664 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d65cff-0b94-4fc0-b44f-5476a3a09f8f admin admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:51.338305 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d65cff-0b94-4fc0-b44f-5476a3a09f8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:51.340243 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d65cff-0b94-4fc0-b44f-5476a3a09f8f admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:51.377798 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 631/1249] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:19:51 2022] POST /identity/v3/projects => generated 393 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:19:51.377798 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef8a537-db0c-4db5-8e23-49ea91877d2a admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:51.432188 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:19:51.435715 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef8a537-db0c-4db5-8e23-49ea91877d2a admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:51.441327 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77ba52f-99b4-4037-b751-fada65e88af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:51.441934 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ba52f-99b4-4037-b751-fada65e88af1 admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:51.442214 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ba52f-99b4-4037-b751-fada65e88af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:51.442486 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ba52f-99b4-4037-b751-fada65e88af1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:51.443105 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77ba52f-99b4-4037-b751-fada65e88af1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:51.445133 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77ba52f-99b4-4037-b751-fada65e88af1 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:51.456699 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 619/1250] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:19:51 2022] POST /identity/v3/projects => generated 405 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:19:51.456699 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d65cff-0b94-4fc0-b44f-5476a3a09f8f admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:51.514283 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d65cff-0b94-4fc0-b44f-5476a3a09f8f admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:51.518752 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc70272b-65a7-4b55-af82-11b8e6f191e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:51.519346 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc70272b-65a7-4b55-af82-11b8e6f191e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:51.519653 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc70272b-65a7-4b55-af82-11b8e6f191e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:51.519919 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc70272b-65a7-4b55-af82-11b8e6f191e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:51.520539 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc70272b-65a7-4b55-af82-11b8e6f191e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:51.522459 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc70272b-65a7-4b55-af82-11b8e6f191e5 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:51.554002 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 632/1251] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:19:51 2022] POST /identity/v3/projects => generated 377 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:19:51.555008 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77ba52f-99b4-4037-b751-fada65e88af1 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:51.615862 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77ba52f-99b4-4037-b751-fada65e88af1 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:51.619793 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 620/1252] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:19:51 2022] POST /identity/v3/projects => generated 379 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:19:51.619793 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc70272b-65a7-4b55-af82-11b8e6f191e5 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:51.648420 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:19:51.675912 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f678b966-7e59-4156-9ca5-6f86c3d1b6a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:51.676806 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f678b966-7e59-4156-9ca5-6f86c3d1b6a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:51.677079 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f678b966-7e59-4156-9ca5-6f86c3d1b6a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:51.677343 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f678b966-7e59-4156-9ca5-6f86c3d1b6a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:51.677964 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f678b966-7e59-4156-9ca5-6f86c3d1b6a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:51.679907 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f678b966-7e59-4156-9ca5-6f86c3d1b6a9 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:51.681602 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc70272b-65a7-4b55-af82-11b8e6f191e5 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:51.686423 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad106ef4-433c-48ac-a585-24e196b47fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:51.687085 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad106ef4-433c-48ac-a585-24e196b47fd6 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:51.687387 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad106ef4-433c-48ac-a585-24e196b47fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:51.687661 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad106ef4-433c-48ac-a585-24e196b47fd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:51.688303 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad106ef4-433c-48ac-a585-24e196b47fd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:51.689977 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad106ef4-433c-48ac-a585-24e196b47fd6 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:51.700577 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 621/1253] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:19:51 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:19:51.701578 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad106ef4-433c-48ac-a585-24e196b47fd6 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:51.710770 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad106ef4-433c-48ac-a585-24e196b47fd6 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:51.715233 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d15f9ed-f975-4b35-ac0e-2a4f7b29694b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:51.715865 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d15f9ed-f975-4b35-ac0e-2a4f7b29694b admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:51.716141 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d15f9ed-f975-4b35-ac0e-2a4f7b29694b admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:51.716401 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d15f9ed-f975-4b35-ac0e-2a4f7b29694b admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:51.717009 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d15f9ed-f975-4b35-ac0e-2a4f7b29694b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:51.719060 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d15f9ed-f975-4b35-ac0e-2a4f7b29694b admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:51.728674 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 622/1254] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:19:51 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:19:51.729567 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d15f9ed-f975-4b35-ac0e-2a4f7b29694b admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:51.739574 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d15f9ed-f975-4b35-ac0e-2a4f7b29694b admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:51.744277 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b12233-5f87-47e2-8d23-32f5bd5d09d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:51.744870 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b12233-5f87-47e2-8d23-32f5bd5d09d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:51.745145 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b12233-5f87-47e2-8d23-32f5bd5d09d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:51.745415 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b12233-5f87-47e2-8d23-32f5bd5d09d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:51.746016 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b12233-5f87-47e2-8d23-32f5bd5d09d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:51.747578 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b12233-5f87-47e2-8d23-32f5bd5d09d1 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:51.757008 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 623/1255] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:19:51 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:19:51.757736 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b12233-5f87-47e2-8d23-32f5bd5d09d1 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:51.767132 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b12233-5f87-47e2-8d23-32f5bd5d09d1 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:51.770853 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 633/1256] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:19:51 2022] POST /identity/v3/projects => generated 375 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:19:51.771619 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f678b966-7e59-4156-9ca5-6f86c3d1b6a9 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:51.771726 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e5ae4f-ba75-490c-8b6c-b2b14a9b6250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:51.772295 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e5ae4f-ba75-490c-8b6c-b2b14a9b6250 admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:51.772515 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e5ae4f-ba75-490c-8b6c-b2b14a9b6250 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:51.772756 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e5ae4f-ba75-490c-8b6c-b2b14a9b6250 admin admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:51.773371 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e5ae4f-ba75-490c-8b6c-b2b14a9b6250 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:51.775286 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e5ae4f-ba75-490c-8b6c-b2b14a9b6250 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:51.833600 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f678b966-7e59-4156-9ca5-6f86c3d1b6a9 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:51.838297 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9533b51-6f7b-4c5d-8ebb-9a375bf268ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:51.838900 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9533b51-6f7b-4c5d-8ebb-9a375bf268ef admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:51.839174 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9533b51-6f7b-4c5d-8ebb-9a375bf268ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:51.839474 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9533b51-6f7b-4c5d-8ebb-9a375bf268ef admin admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:51.840096 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9533b51-6f7b-4c5d-8ebb-9a375bf268ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:51.841112 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: WARNING py.warnings [None req-c9533b51-6f7b-4c5d-8ebb-9a375bf268ef admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 12 00:19:51.841112 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: warnings.warn(msg) Oct 12 00:19:51.841112 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: Oct 12 00:19:51.842347 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9533b51-6f7b-4c5d-8ebb-9a375bf268ef admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:51.850302 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 624/1257] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:19:51 2022] POST /identity/v3/users => generated 297 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:19:51.850302 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e5ae4f-ba75-490c-8b6c-b2b14a9b6250 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:51.859784 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e5ae4f-ba75-490c-8b6c-b2b14a9b6250 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:51.864532 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd845224-172c-4d3f-8e52-536fc3d339ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:51.865115 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd845224-172c-4d3f-8e52-536fc3d339ac admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:51.866627 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd845224-172c-4d3f-8e52-536fc3d339ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:51.866627 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd845224-172c-4d3f-8e52-536fc3d339ac admin admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:51.866627 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd845224-172c-4d3f-8e52-536fc3d339ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:51.868158 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd845224-172c-4d3f-8e52-536fc3d339ac admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:51.900547 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 634/1258] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:19:51 2022] POST /identity/v3/users => generated 303 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:19:51.901401 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9533b51-6f7b-4c5d-8ebb-9a375bf268ef admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:51.907809 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:19:51.911517 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9533b51-6f7b-4c5d-8ebb-9a375bf268ef admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:51.916083 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b3ed9f-afce-456a-bd15-00a8a945bbbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:51.916649 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b3ed9f-afce-456a-bd15-00a8a945bbbf admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:51.916924 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b3ed9f-afce-456a-bd15-00a8a945bbbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:51.917192 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b3ed9f-afce-456a-bd15-00a8a945bbbf admin admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:51.917796 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b3ed9f-afce-456a-bd15-00a8a945bbbf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:51.919772 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b3ed9f-afce-456a-bd15-00a8a945bbbf admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:51.961281 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 625/1259] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:19:51 2022] POST /identity/v3/users => generated 289 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:19:51.961281 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd845224-172c-4d3f-8e52-536fc3d339ac admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:51.969435 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd845224-172c-4d3f-8e52-536fc3d339ac admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:51.973953 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5650cb56-b790-4329-9e23-cc76ffd42ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:51.974573 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5650cb56-b790-4329-9e23-cc76ffd42ef6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:51.975765 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5650cb56-b790-4329-9e23-cc76ffd42ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:51.975765 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5650cb56-b790-4329-9e23-cc76ffd42ef6 admin admin] PATH_INFO: `/v3/projects/5175428cacd149c792aff29b84c5631a/users/d1bb3b1ab7ac4abca069396a138a61e5/roles/ef689f96442945e8a7bc8cbfee13d909` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:51.990255 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5650cb56-b790-4329-9e23-cc76ffd42ef6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5175428cacd149c792aff29b84c5631a, user_id=d1bb3b1ab7ac4abca069396a138a61e5, role_id=ef689f96442945e8a7bc8cbfee13d909)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:51.992922 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5650cb56-b790-4329-9e23-cc76ffd42ef6 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:52.033272 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 635/1260] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:19:51 2022] POST /identity/v3/users => generated 290 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:19:52.033272 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b3ed9f-afce-456a-bd15-00a8a945bbbf admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:52.040443 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b3ed9f-afce-456a-bd15-00a8a945bbbf admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:52.045010 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ded867a-61d1-4285-a58f-51b6e31f6aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:52.045638 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ded867a-61d1-4285-a58f-51b6e31f6aa3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:52.045918 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ded867a-61d1-4285-a58f-51b6e31f6aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:52.046191 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ded867a-61d1-4285-a58f-51b6e31f6aa3 admin admin] PATH_INFO: `/v3/projects/10160596b45c49249155b0c2585e70bb/users/de766cb582ba4c18b1fda9bf4ad43f65/roles/ef689f96442945e8a7bc8cbfee13d909` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:52.062363 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ded867a-61d1-4285-a58f-51b6e31f6aa3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10160596b45c49249155b0c2585e70bb, user_id=de766cb582ba4c18b1fda9bf4ad43f65, role_id=ef689f96442945e8a7bc8cbfee13d909)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:52.065106 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ded867a-61d1-4285-a58f-51b6e31f6aa3 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:52.101015 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 626/1261] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:19:51 2022] PUT /identity/v3/projects/5175428cacd149c792aff29b84c5631a/users/d1bb3b1ab7ac4abca069396a138a61e5/roles/ef689f96442945e8a7bc8cbfee13d909 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:19:52.110541 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5650cb56-b790-4329-9e23-cc76ffd42ef6 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:52.123535 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 636/1262] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:19:52 2022] PUT /identity/v3/projects/10160596b45c49249155b0c2585e70bb/users/de766cb582ba4c18b1fda9bf4ad43f65/roles/ef689f96442945e8a7bc8cbfee13d909 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:19:52.126961 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ded867a-61d1-4285-a58f-51b6e31f6aa3 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:52.165412 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5650cb56-b790-4329-9e23-cc76ffd42ef6 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:52.186250 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ded867a-61d1-4285-a58f-51b6e31f6aa3 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:52.192000 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8e2833-af9e-4bea-b6f4-d892150f5978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:52.192804 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e2833-af9e-4bea-b6f4-d892150f5978 admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:52.193228 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e2833-af9e-4bea-b6f4-d892150f5978 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:52.193783 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e2833-af9e-4bea-b6f4-d892150f5978 admin admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:52.194566 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8e2833-af9e-4bea-b6f4-d892150f5978 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:52.196502 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8e2833-af9e-4bea-b6f4-d892150f5978 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:52.210622 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:19:52.222209 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da6c776-ea53-4813-94e9-9b9f53f3a351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:52.222819 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da6c776-ea53-4813-94e9-9b9f53f3a351 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:52.223095 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da6c776-ea53-4813-94e9-9b9f53f3a351 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:52.223383 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da6c776-ea53-4813-94e9-9b9f53f3a351 admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:52.224033 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da6c776-ea53-4813-94e9-9b9f53f3a351 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:52.225618 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da6c776-ea53-4813-94e9-9b9f53f3a351 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:52.235003 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 627/1263] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:19:52 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:19:52.235825 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da6c776-ea53-4813-94e9-9b9f53f3a351 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:52.245256 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da6c776-ea53-4813-94e9-9b9f53f3a351 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:52.250296 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbac2245-bdfd-4145-8bd0-36dbf6b6f6fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:52.250877 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbac2245-bdfd-4145-8bd0-36dbf6b6f6fc admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:52.251759 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbac2245-bdfd-4145-8bd0-36dbf6b6f6fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:52.251759 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbac2245-bdfd-4145-8bd0-36dbf6b6f6fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:52.252055 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbac2245-bdfd-4145-8bd0-36dbf6b6f6fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:52.253625 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbac2245-bdfd-4145-8bd0-36dbf6b6f6fc admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:52.263093 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 628/1264] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:19:52 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:19:52.264127 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbac2245-bdfd-4145-8bd0-36dbf6b6f6fc admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:52.273739 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbac2245-bdfd-4145-8bd0-36dbf6b6f6fc admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:52.278278 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c3abf4-64fc-482f-9a08-f6b63aa7de99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:52.278820 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c3abf4-64fc-482f-9a08-f6b63aa7de99 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:52.279094 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c3abf4-64fc-482f-9a08-f6b63aa7de99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:52.279384 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c3abf4-64fc-482f-9a08-f6b63aa7de99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:52.279989 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c3abf4-64fc-482f-9a08-f6b63aa7de99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:52.281544 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c3abf4-64fc-482f-9a08-f6b63aa7de99 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:52.291594 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 629/1265] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:19:52 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:19:52.292303 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c3abf4-64fc-482f-9a08-f6b63aa7de99 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:52.301327 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c3abf4-64fc-482f-9a08-f6b63aa7de99 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:52.305800 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a187c17-2ffc-4ef1-9e65-9d0d116eddec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:52.306357 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a187c17-2ffc-4ef1-9e65-9d0d116eddec admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:52.306634 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a187c17-2ffc-4ef1-9e65-9d0d116eddec admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:52.306922 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a187c17-2ffc-4ef1-9e65-9d0d116eddec admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:52.307542 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a187c17-2ffc-4ef1-9e65-9d0d116eddec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:52.309068 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a187c17-2ffc-4ef1-9e65-9d0d116eddec admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:52.318856 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 630/1266] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:19:52 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:19:52.319636 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a187c17-2ffc-4ef1-9e65-9d0d116eddec admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:52.329201 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 637/1267] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:19:52 2022] POST /identity/v3/users => generated 288 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:19:52.332330 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e49e48a-b04b-43fe-8a5b-49280f390310 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:52.332330 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e49e48a-b04b-43fe-8a5b-49280f390310 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:52.332330 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e49e48a-b04b-43fe-8a5b-49280f390310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:52.332773 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a187c17-2ffc-4ef1-9e65-9d0d116eddec admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:52.336623 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac086e9-55e6-403b-8d8f-1b971a0da069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:52.337373 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac086e9-55e6-403b-8d8f-1b971a0da069 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:52.337675 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac086e9-55e6-403b-8d8f-1b971a0da069 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:52.337947 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac086e9-55e6-403b-8d8f-1b971a0da069 admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:52.338567 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac086e9-55e6-403b-8d8f-1b971a0da069 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:52.340184 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac086e9-55e6-403b-8d8f-1b971a0da069 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:52.349985 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 631/1268] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:19:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:19:52.351158 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63703f-b70e-4eb5-87c2-0811cf829c65 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:52.351458 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63703f-b70e-4eb5-87c2-0811cf829c65 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:52.351729 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63703f-b70e-4eb5-87c2-0811cf829c65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:52.358347 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-1e49e48a-b04b-43fe-8a5b-49280f390310 None None] MFA Rules not processed for user `d1bb3b1ab7ac4abca069396a138a61e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:19:52.359695 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-1e49e48a-b04b-43fe-8a5b-49280f390310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:19:52.370361 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-8e63703f-b70e-4eb5-87c2-0811cf829c65 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:19:52.371645 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-8e63703f-b70e-4eb5-87c2-0811cf829c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:19:52.380219 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 632/1269] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:19:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:19:52.383823 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e63703f-b70e-4eb5-87c2-0811cf829c65 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:52.397720 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e63703f-b70e-4eb5-87c2-0811cf829c65 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:52.402654 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13a581e-09e7-47a4-b29e-7c395f7a27ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:52.403316 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13a581e-09e7-47a4-b29e-7c395f7a27ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:52.403629 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13a581e-09e7-47a4-b29e-7c395f7a27ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:52.403898 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13a581e-09e7-47a4-b29e-7c395f7a27ad admin admin] PATH_INFO: `/v3/projects/54567f4ea839415a94397e13f93d71cc/users/bca53aca7b134df1949deb10858f14d0/roles/ef689f96442945e8a7bc8cbfee13d909` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:52.418331 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13a581e-09e7-47a4-b29e-7c395f7a27ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=54567f4ea839415a94397e13f93d71cc, user_id=bca53aca7b134df1949deb10858f14d0, role_id=ef689f96442945e8a7bc8cbfee13d909)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:52.421004 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13a581e-09e7-47a4-b29e-7c395f7a27ad admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:52.430987 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 638/1270] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:19:52 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:19:52.431848 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e49e48a-b04b-43fe-8a5b-49280f390310 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:52.442111 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e49e48a-b04b-43fe-8a5b-49280f390310 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:52.447698 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d022ee-f24d-4a3f-b217-ad777787d173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:52.447698 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d022ee-f24d-4a3f-b217-ad777787d173 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:52.447698 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d022ee-f24d-4a3f-b217-ad777787d173 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:52.448007 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d022ee-f24d-4a3f-b217-ad777787d173 admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:52.448538 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d022ee-f24d-4a3f-b217-ad777787d173 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:52.450117 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d022ee-f24d-4a3f-b217-ad777787d173 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:52.460487 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 639/1271] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:19:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:19:52.461237 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d022ee-f24d-4a3f-b217-ad777787d173 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:52.470467 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d022ee-f24d-4a3f-b217-ad777787d173 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:52.474971 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a35d3c-5544-4c6b-aad1-cc8d7676f869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:52.475545 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a35d3c-5544-4c6b-aad1-cc8d7676f869 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:52.475820 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a35d3c-5544-4c6b-aad1-cc8d7676f869 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:52.476086 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a35d3c-5544-4c6b-aad1-cc8d7676f869 admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:52.476726 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a35d3c-5544-4c6b-aad1-cc8d7676f869 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:52.478229 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a35d3c-5544-4c6b-aad1-cc8d7676f869 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:52.487687 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 640/1272] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:19:52 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:19:52.488486 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a35d3c-5544-4c6b-aad1-cc8d7676f869 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:52.496045 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 633/1273] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:19:52 2022] PUT /identity/v3/projects/54567f4ea839415a94397e13f93d71cc/users/bca53aca7b134df1949deb10858f14d0/roles/ef689f96442945e8a7bc8cbfee13d909 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:19:52.496920 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13a581e-09e7-47a4-b29e-7c395f7a27ad admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:52.498285 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a35d3c-5544-4c6b-aad1-cc8d7676f869 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:52.517078 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:19:52.558294 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8f45e7-d086-485a-8242-b7b3465ac969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:52.558837 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8f45e7-d086-485a-8242-b7b3465ac969 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:52.559203 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8f45e7-d086-485a-8242-b7b3465ac969 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:52.559522 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8f45e7-d086-485a-8242-b7b3465ac969 admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:52.560159 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8f45e7-d086-485a-8242-b7b3465ac969 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:52.561781 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8f45e7-d086-485a-8242-b7b3465ac969 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:52.564757 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13a581e-09e7-47a4-b29e-7c395f7a27ad admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:52.569481 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4102027b-a1f5-434c-852b-5ced488fc0d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:52.570091 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4102027b-a1f5-434c-852b-5ced488fc0d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:52.570422 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4102027b-a1f5-434c-852b-5ced488fc0d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:52.570635 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4102027b-a1f5-434c-852b-5ced488fc0d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:52.571298 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4102027b-a1f5-434c-852b-5ced488fc0d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:52.572542 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 641/1274] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:19:52 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:19:52.572908 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4102027b-a1f5-434c-852b-5ced488fc0d7 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:52.573417 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8f45e7-d086-485a-8242-b7b3465ac969 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:52.583575 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8f45e7-d086-485a-8242-b7b3465ac969 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:52.585379 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 634/1275] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:19:52 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:19:52.586187 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4102027b-a1f5-434c-852b-5ced488fc0d7 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:52.588496 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a14fbda-043c-47fa-9f13-1c38ec409027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:52.589105 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a14fbda-043c-47fa-9f13-1c38ec409027 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:52.589382 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a14fbda-043c-47fa-9f13-1c38ec409027 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:52.589651 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a14fbda-043c-47fa-9f13-1c38ec409027 admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:52.590279 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a14fbda-043c-47fa-9f13-1c38ec409027 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:52.592037 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a14fbda-043c-47fa-9f13-1c38ec409027 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:52.602572 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 642/1276] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:19:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:19:52.603406 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a14fbda-043c-47fa-9f13-1c38ec409027 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:52.613578 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a14fbda-043c-47fa-9f13-1c38ec409027 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:52.619045 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93aecdd-c37d-44f0-81c3-5d1a4f99aa8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:52.619774 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93aecdd-c37d-44f0-81c3-5d1a4f99aa8d admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:52.620069 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93aecdd-c37d-44f0-81c3-5d1a4f99aa8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:52.620348 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93aecdd-c37d-44f0-81c3-5d1a4f99aa8d admin admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:52.620971 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93aecdd-c37d-44f0-81c3-5d1a4f99aa8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:52.622851 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93aecdd-c37d-44f0-81c3-5d1a4f99aa8d admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:52.640579 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4102027b-a1f5-434c-852b-5ced488fc0d7 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:52.644199 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faa6471-10bb-474f-b7f3-e63ce084f76a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:52.644799 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faa6471-10bb-474f-b7f3-e63ce084f76a service neutron] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:52.645074 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faa6471-10bb-474f-b7f3-e63ce084f76a service neutron] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:52.645342 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faa6471-10bb-474f-b7f3-e63ce084f76a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:52.654236 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faa6471-10bb-474f-b7f3-e63ce084f76a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:52.655695 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faa6471-10bb-474f-b7f3-e63ce084f76a service neutron] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:52.665057 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 635/1277] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:19:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:19:52.665802 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faa6471-10bb-474f-b7f3-e63ce084f76a service neutron] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:52.675094 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faa6471-10bb-474f-b7f3-e63ce084f76a service neutron] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:52.679876 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cf2aa2-91c9-4a7f-8a6d-7b7375e07d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:52.680512 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cf2aa2-91c9-4a7f-8a6d-7b7375e07d37 admin admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:52.680789 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cf2aa2-91c9-4a7f-8a6d-7b7375e07d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:52.681085 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cf2aa2-91c9-4a7f-8a6d-7b7375e07d37 admin admin] PATH_INFO: `/v3/projects/38c7ea1f3f9848b484a56cf74a2bd129/users/cdd87151f3af4a3bb1c3815db5d0e8fa/roles/ef689f96442945e8a7bc8cbfee13d909` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:52.684285 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:19:52.699644 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 643/1278] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:19:52 2022] POST /identity/v3/projects => generated 365 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:19:52.699828 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cf2aa2-91c9-4a7f-8a6d-7b7375e07d37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38c7ea1f3f9848b484a56cf74a2bd129, user_id=cdd87151f3af4a3bb1c3815db5d0e8fa, role_id=ef689f96442945e8a7bc8cbfee13d909)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:52.699958 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93aecdd-c37d-44f0-81c3-5d1a4f99aa8d admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:52.700050 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cf2aa2-91c9-4a7f-8a6d-7b7375e07d37 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:52.758127 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93aecdd-c37d-44f0-81c3-5d1a4f99aa8d admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:52.766476 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 636/1279] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:19:52 2022] PUT /identity/v3/projects/38c7ea1f3f9848b484a56cf74a2bd129/users/cdd87151f3af4a3bb1c3815db5d0e8fa/roles/ef689f96442945e8a7bc8cbfee13d909 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:19:52.766476 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ede5cd-9aa0-4c7c-ae8f-10f1fd9df91b None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:52.766476 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ede5cd-9aa0-4c7c-ae8f-10f1fd9df91b None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:52.766476 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ede5cd-9aa0-4c7c-ae8f-10f1fd9df91b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:52.785214 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-d2ede5cd-9aa0-4c7c-ae8f-10f1fd9df91b None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:19:52.789574 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-d2ede5cd-9aa0-4c7c-ae8f-10f1fd9df91b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:19:52.828773 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec11ccb1-fc35-45e0-92df-e219cc53a04c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:52.830314 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec11ccb1-fc35-45e0-92df-e219cc53a04c admin admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:52.830765 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec11ccb1-fc35-45e0-92df-e219cc53a04c admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:52.831557 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec11ccb1-fc35-45e0-92df-e219cc53a04c admin admin] PATH_INFO: `/v3/projects/a73b565e3ac34231a7df03912dcf74fb/users/0d0276e935f247a9955fe576224821b4/roles/ef689f96442945e8a7bc8cbfee13d909` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:52.848619 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec11ccb1-fc35-45e0-92df-e219cc53a04c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a73b565e3ac34231a7df03912dcf74fb, user_id=0d0276e935f247a9955fe576224821b4, role_id=ef689f96442945e8a7bc8cbfee13d909)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:52.851605 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec11ccb1-fc35-45e0-92df-e219cc53a04c admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:52.852164 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 637/1280] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:19:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:19:52.853046 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ede5cd-9aa0-4c7c-ae8f-10f1fd9df91b None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:52.867488 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ede5cd-9aa0-4c7c-ae8f-10f1fd9df91b None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:52.872241 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfea6161-cd92-45ff-85f5-4315d786f491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:52.872908 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfea6161-cd92-45ff-85f5-4315d786f491 admin admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:52.873182 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfea6161-cd92-45ff-85f5-4315d786f491 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:52.873452 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfea6161-cd92-45ff-85f5-4315d786f491 admin admin] PATH_INFO: `/v3/projects/bbfafa65b93241bcb62d41018f6ccd1f/users/b4dba90d485945e1821be4667d5a5bc9/roles/ef689f96442945e8a7bc8cbfee13d909` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:52.888421 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfea6161-cd92-45ff-85f5-4315d786f491 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbfafa65b93241bcb62d41018f6ccd1f, user_id=b4dba90d485945e1821be4667d5a5bc9, role_id=ef689f96442945e8a7bc8cbfee13d909)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:52.891342 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfea6161-cd92-45ff-85f5-4315d786f491 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:52.942400 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 644/1281] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:19:52 2022] PUT /identity/v3/projects/a73b565e3ac34231a7df03912dcf74fb/users/0d0276e935f247a9955fe576224821b4/roles/ef689f96442945e8a7bc8cbfee13d909 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:19:52.951670 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec11ccb1-fc35-45e0-92df-e219cc53a04c admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:52.957036 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:19:53.004533 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 638/1282] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:19:52 2022] PUT /identity/v3/projects/bbfafa65b93241bcb62d41018f6ccd1f/users/b4dba90d485945e1821be4667d5a5bc9/roles/ef689f96442945e8a7bc8cbfee13d909 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:19:53.009058 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfea6161-cd92-45ff-85f5-4315d786f491 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:53.010976 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec11ccb1-fc35-45e0-92df-e219cc53a04c admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:53.071088 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cba5f5-23ba-44ef-81cb-7bef8b982033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:53.071736 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cba5f5-23ba-44ef-81cb-7bef8b982033 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:53.072048 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cba5f5-23ba-44ef-81cb-7bef8b982033 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:53.072323 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cba5f5-23ba-44ef-81cb-7bef8b982033 admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:53.073176 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cba5f5-23ba-44ef-81cb-7bef8b982033 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:53.074784 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cba5f5-23ba-44ef-81cb-7bef8b982033 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:53.075124 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfea6161-cd92-45ff-85f5-4315d786f491 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:53.079880 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d35e5a-3479-4a9e-b411-bea21015a7df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:53.080484 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d35e5a-3479-4a9e-b411-bea21015a7df admin admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:53.080761 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d35e5a-3479-4a9e-b411-bea21015a7df admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:53.081032 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d35e5a-3479-4a9e-b411-bea21015a7df admin admin] PATH_INFO: `/v3/projects/a963d5b895ca4aa3abd1406893e4b572/users/e1bf651b8858485392bd2dcfec738e2c/roles/ef689f96442945e8a7bc8cbfee13d909` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:53.084711 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 645/1283] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:19:52 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:19:53.085486 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cba5f5-23ba-44ef-81cb-7bef8b982033 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:53.096882 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d35e5a-3479-4a9e-b411-bea21015a7df admin admin] RBAC: Authorizing `identity:create_grant(project_id=a963d5b895ca4aa3abd1406893e4b572, user_id=e1bf651b8858485392bd2dcfec738e2c, role_id=ef689f96442945e8a7bc8cbfee13d909)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:53.097055 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cba5f5-23ba-44ef-81cb-7bef8b982033 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:53.100861 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d35e5a-3479-4a9e-b411-bea21015a7df admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:53.103552 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ea078e-a111-4ddc-a8ec-ca2047cc1b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:53.104174 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ea078e-a111-4ddc-a8ec-ca2047cc1b5f admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:53.104454 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ea078e-a111-4ddc-a8ec-ca2047cc1b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:53.104731 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ea078e-a111-4ddc-a8ec-ca2047cc1b5f admin admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:53.105369 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ea078e-a111-4ddc-a8ec-ca2047cc1b5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:53.107359 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ea078e-a111-4ddc-a8ec-ca2047cc1b5f admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:53.183007 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 639/1284] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:19:53 2022] PUT /identity/v3/projects/a963d5b895ca4aa3abd1406893e4b572/users/e1bf651b8858485392bd2dcfec738e2c/roles/ef689f96442945e8a7bc8cbfee13d909 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:19:53.183007 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f05cc38-2e8d-438b-be8d-8fa34b2fd638 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:53.183007 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f05cc38-2e8d-438b-be8d-8fa34b2fd638 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:53.183007 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f05cc38-2e8d-438b-be8d-8fa34b2fd638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:53.195728 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-0f05cc38-2e8d-438b-be8d-8fa34b2fd638 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:19:53.196926 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-0f05cc38-2e8d-438b-be8d-8fa34b2fd638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:19:53.239999 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 646/1285] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:19:53 2022] POST /identity/v3/users => generated 289 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:19:53.241240 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ea078e-a111-4ddc-a8ec-ca2047cc1b5f admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:53.247509 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:19:53.267058 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 640/1286] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:19:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:19:53.268481 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670872d0-8bf7-4487-b792-3e0d2a470768 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:53.273352 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670872d0-8bf7-4487-b792-3e0d2a470768 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:53.273352 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670872d0-8bf7-4487-b792-3e0d2a470768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:53.302500 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ea078e-a111-4ddc-a8ec-ca2047cc1b5f admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:53.306024 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-670872d0-8bf7-4487-b792-3e0d2a470768 None None] MFA Rules not processed for user `0d0276e935f247a9955fe576224821b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:19:53.307350 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-670872d0-8bf7-4487-b792-3e0d2a470768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:19:53.353767 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b239c3c-e01b-4093-bff7-a97abe4801a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:53.354345 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b239c3c-e01b-4093-bff7-a97abe4801a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:53.354625 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b239c3c-e01b-4093-bff7-a97abe4801a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:53.354917 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b239c3c-e01b-4093-bff7-a97abe4801a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:53.364659 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b239c3c-e01b-4093-bff7-a97abe4801a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:53.366201 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b239c3c-e01b-4093-bff7-a97abe4801a3 service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:53.370539 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:19:53.376983 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 641/1287] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:19:53 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:19:53.379186 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7054df8-b12e-49b6-a73f-f66345245634 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:53.380012 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7054df8-b12e-49b6-a73f-f66345245634 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:53.382464 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7054df8-b12e-49b6-a73f-f66345245634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:53.382572 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 647/1288] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:19:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:19:53.382572 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b239c3c-e01b-4093-bff7-a97abe4801a3 service neutron] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:53.392467 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b239c3c-e01b-4093-bff7-a97abe4801a3 service neutron] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:53.397783 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015c1198-22d7-4e8e-9ce3-82b2b0ff4e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:53.398456 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015c1198-22d7-4e8e-9ce3-82b2b0ff4e20 admin admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:53.398735 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015c1198-22d7-4e8e-9ce3-82b2b0ff4e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:53.399009 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015c1198-22d7-4e8e-9ce3-82b2b0ff4e20 admin admin] PATH_INFO: `/v3/projects/532f356fff2a4e8db81955a7071bdb9b/users/fb2ff6db932541aaaa90bbec30091aa0/roles/ef689f96442945e8a7bc8cbfee13d909` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:53.401563 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-d7054df8-b12e-49b6-a73f-f66345245634 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:19:53.405482 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-d7054df8-b12e-49b6-a73f-f66345245634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:19:53.411886 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 642/1289] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:19:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:19:53.412986 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1606eadf-c927-44ac-9064-dbde980e15a8 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:53.413340 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1606eadf-c927-44ac-9064-dbde980e15a8 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:53.413676 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1606eadf-c927-44ac-9064-dbde980e15a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:53.434807 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015c1198-22d7-4e8e-9ce3-82b2b0ff4e20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=532f356fff2a4e8db81955a7071bdb9b, user_id=fb2ff6db932541aaaa90bbec30091aa0, role_id=ef689f96442945e8a7bc8cbfee13d909)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:53.438366 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015c1198-22d7-4e8e-9ce3-82b2b0ff4e20 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:53.444749 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-1606eadf-c927-44ac-9064-dbde980e15a8 None None] MFA Rules not processed for user `e1bf651b8858485392bd2dcfec738e2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:19:53.445964 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-1606eadf-c927-44ac-9064-dbde980e15a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:19:53.513379 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 643/1290] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:19:53 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:19:53.514537 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1606eadf-c927-44ac-9064-dbde980e15a8 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:53.527917 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 648/1291] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:19:53 2022] PUT /identity/v3/projects/532f356fff2a4e8db81955a7071bdb9b/users/fb2ff6db932541aaaa90bbec30091aa0/roles/ef689f96442945e8a7bc8cbfee13d909 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:19:53.527917 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015c1198-22d7-4e8e-9ce3-82b2b0ff4e20 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:53.534798 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1606eadf-c927-44ac-9064-dbde980e15a8 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:53.604492 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:19:53.608223 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015c1198-22d7-4e8e-9ce3-82b2b0ff4e20 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:53.610297 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c1c866-26f2-472c-abd3-d2c51588061b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:53.611012 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c1c866-26f2-472c-abd3-d2c51588061b admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:53.611441 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c1c866-26f2-472c-abd3-d2c51588061b admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:53.611842 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c1c866-26f2-472c-abd3-d2c51588061b admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:53.612491 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c1c866-26f2-472c-abd3-d2c51588061b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:53.612940 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd9e662-3539-4b61-96e6-ee8e3c409de6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:53.613545 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd9e662-3539-4b61-96e6-ee8e3c409de6 service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:53.613844 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd9e662-3539-4b61-96e6-ee8e3c409de6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:53.614090 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd9e662-3539-4b61-96e6-ee8e3c409de6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:53.614194 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c1c866-26f2-472c-abd3-d2c51588061b admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:53.623638 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd9e662-3539-4b61-96e6-ee8e3c409de6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:53.624950 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 644/1292] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:19:53 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:19:53.625156 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd9e662-3539-4b61-96e6-ee8e3c409de6 service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:53.625959 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c1c866-26f2-472c-abd3-d2c51588061b admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:53.635041 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 649/1293] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:19:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:19:53.635887 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd9e662-3539-4b61-96e6-ee8e3c409de6 service neutron] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:53.641326 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c1c866-26f2-472c-abd3-d2c51588061b admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:53.645082 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd9e662-3539-4b61-96e6-ee8e3c409de6 service neutron] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:53.646842 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee653c1-6d0c-431c-a510-2b5e685df083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:53.647440 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee653c1-6d0c-431c-a510-2b5e685df083 admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:53.650849 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee653c1-6d0c-431c-a510-2b5e685df083 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:53.650849 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee653c1-6d0c-431c-a510-2b5e685df083 admin admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:53.650849 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee653c1-6d0c-431c-a510-2b5e685df083 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:53.651231 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f889ef-1ad0-4007-928f-397984246a8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:53.651231 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f889ef-1ad0-4007-928f-397984246a8a service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:53.651231 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f889ef-1ad0-4007-928f-397984246a8a service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:53.651231 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f889ef-1ad0-4007-928f-397984246a8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:53.651818 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee653c1-6d0c-431c-a510-2b5e685df083 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:53.660435 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f889ef-1ad0-4007-928f-397984246a8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:53.662101 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f889ef-1ad0-4007-928f-397984246a8a service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:53.672431 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 650/1294] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:19:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:19:53.676371 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f889ef-1ad0-4007-928f-397984246a8a service neutron] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:53.686251 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f889ef-1ad0-4007-928f-397984246a8a service neutron] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:53.692015 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e981ab-5ecb-419a-a58d-9adc2900e21c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:53.693191 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e981ab-5ecb-419a-a58d-9adc2900e21c admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:53.693191 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e981ab-5ecb-419a-a58d-9adc2900e21c admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:53.693191 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e981ab-5ecb-419a-a58d-9adc2900e21c admin admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:53.693768 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e981ab-5ecb-419a-a58d-9adc2900e21c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:53.695819 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e981ab-5ecb-419a-a58d-9adc2900e21c admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:53.767756 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 645/1295] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:19:53 2022] POST /identity/v3/projects => generated 403 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:19:53.767756 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3806d2aa-d21c-4520-bbd2-36bee2277ba8 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:53.767756 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3806d2aa-d21c-4520-bbd2-36bee2277ba8 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:53.767756 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3806d2aa-d21c-4520-bbd2-36bee2277ba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:53.794840 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-3806d2aa-d21c-4520-bbd2-36bee2277ba8 None None] MFA Rules not processed for user `fb2ff6db932541aaaa90bbec30091aa0`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:19:53.795681 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-3806d2aa-d21c-4520-bbd2-36bee2277ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:19:53.822785 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 651/1296] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:19:53 2022] POST /identity/v3/projects => generated 379 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:19:53.822785 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e981ab-5ecb-419a-a58d-9adc2900e21c admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:53.911863 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e981ab-5ecb-419a-a58d-9adc2900e21c admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:53.918009 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 646/1297] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:19:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:19:53.918905 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3806d2aa-d21c-4520-bbd2-36bee2277ba8 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:53.923609 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f13397-4dd5-437d-9516-e5bf47f158cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:53.924174 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f13397-4dd5-437d-9516-e5bf47f158cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:53.924456 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f13397-4dd5-437d-9516-e5bf47f158cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:53.924733 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f13397-4dd5-437d-9516-e5bf47f158cd admin admin] PATH_INFO: `/v3/projects/fee2b5f3f77f44bf9d929d752aa351c3/users/ab89c78210e246c68e33d23bf97a093f/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:53.945022 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f13397-4dd5-437d-9516-e5bf47f158cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=fee2b5f3f77f44bf9d929d752aa351c3, user_id=ab89c78210e246c68e33d23bf97a093f, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:53.948171 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f13397-4dd5-437d-9516-e5bf47f158cd admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:53.994442 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3806d2aa-d21c-4520-bbd2-36bee2277ba8 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:54.000614 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6463f610-447d-4601-b680-7f7b6187366c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:54.001637 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6463f610-447d-4601-b680-7f7b6187366c admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:54.001995 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6463f610-447d-4601-b680-7f7b6187366c admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:54.002377 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6463f610-447d-4601-b680-7f7b6187366c admin admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:54.003382 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6463f610-447d-4601-b680-7f7b6187366c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:54.005424 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6463f610-447d-4601-b680-7f7b6187366c admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:54.022668 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 652/1298] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:19:53 2022] PUT /identity/v3/projects/fee2b5f3f77f44bf9d929d752aa351c3/users/ab89c78210e246c68e33d23bf97a093f/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:19:54.023586 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f13397-4dd5-437d-9516-e5bf47f158cd admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:54.057137 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:19:54.103117 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f13397-4dd5-437d-9516-e5bf47f158cd admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:54.116259 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e104f04b-561b-4e14-bc3d-8a1841f3a1e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:54.117267 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e104f04b-561b-4e14-bc3d-8a1841f3a1e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:54.117736 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e104f04b-561b-4e14-bc3d-8a1841f3a1e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:54.118426 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e104f04b-561b-4e14-bc3d-8a1841f3a1e7 admin admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:54.118891 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 647/1299] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:19:53 2022] POST /identity/v3/users => generated 308 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:19:54.120462 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e104f04b-561b-4e14-bc3d-8a1841f3a1e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:54.120633 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5ed875-17a3-4677-80f1-7d2848345e9a None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:54.120933 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5ed875-17a3-4677-80f1-7d2848345e9a None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:54.121211 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5ed875-17a3-4677-80f1-7d2848345e9a None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:54.129032 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 648/1300] 10.222.0.36 () {54 vars in 961 bytes} [Wed Oct 12 00:19:54 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:19:54.129908 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5ed875-17a3-4677-80f1-7d2848345e9a None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:54.138657 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e104f04b-561b-4e14-bc3d-8a1841f3a1e7 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:54.154487 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5ed875-17a3-4677-80f1-7d2848345e9a None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:54.166772 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe0a15c-6023-4a49-8ed1-fadf8a652c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:54.167281 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe0a15c-6023-4a49-8ed1-fadf8a652c4b admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:54.167593 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe0a15c-6023-4a49-8ed1-fadf8a652c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:54.172343 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe0a15c-6023-4a49-8ed1-fadf8a652c4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:54.174133 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe0a15c-6023-4a49-8ed1-fadf8a652c4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:54.176733 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe0a15c-6023-4a49-8ed1-fadf8a652c4b admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:54.193682 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 649/1301] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:19:54 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:19:54.195117 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe0a15c-6023-4a49-8ed1-fadf8a652c4b admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:54.208691 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe0a15c-6023-4a49-8ed1-fadf8a652c4b admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:54.214117 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36463787-a07b-49ae-93ed-3f078736d09c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:54.214938 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36463787-a07b-49ae-93ed-3f078736d09c admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:54.215235 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36463787-a07b-49ae-93ed-3f078736d09c admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:54.215537 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36463787-a07b-49ae-93ed-3f078736d09c admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:54.216327 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36463787-a07b-49ae-93ed-3f078736d09c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:54.219762 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36463787-a07b-49ae-93ed-3f078736d09c admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:54.241517 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 650/1302] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:19:54 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:19:54.243921 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851adfed-14d3-4602-94ee-3048ef9f4982 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:54.244430 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851adfed-14d3-4602-94ee-3048ef9f4982 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:54.245280 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851adfed-14d3-4602-94ee-3048ef9f4982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:54.245756 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 653/1303] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:19:54 2022] POST /identity/v3/users => generated 296 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:19:54.259958 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e104f04b-561b-4e14-bc3d-8a1841f3a1e7 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:54.291187 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e104f04b-561b-4e14-bc3d-8a1841f3a1e7 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:54.300861 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b8b1cd-dde3-4872-b7f3-6c2bdac553eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:54.303131 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b8b1cd-dde3-4872-b7f3-6c2bdac553eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:54.303473 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b8b1cd-dde3-4872-b7f3-6c2bdac553eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:54.305636 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b8b1cd-dde3-4872-b7f3-6c2bdac553eb admin admin] PATH_INFO: `/v3/projects/fee2b5f3f77f44bf9d929d752aa351c3/users/ab89c78210e246c68e33d23bf97a093f/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:54.307568 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b8b1cd-dde3-4872-b7f3-6c2bdac553eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=fee2b5f3f77f44bf9d929d752aa351c3, user_id=ab89c78210e246c68e33d23bf97a093f, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:54.312962 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b8b1cd-dde3-4872-b7f3-6c2bdac553eb admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:54.334336 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-851adfed-14d3-4602-94ee-3048ef9f4982 None None] MFA Rules not processed for user `a515b75aa3944912ad2364ffd7924f40`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:19:54.335880 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-851adfed-14d3-4602-94ee-3048ef9f4982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:19:54.336897 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 654/1304] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:19:54 2022] PUT /identity/v3/projects/fee2b5f3f77f44bf9d929d752aa351c3/users/ab89c78210e246c68e33d23bf97a093f/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:19:54.338000 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b8b1cd-dde3-4872-b7f3-6c2bdac553eb admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:54.367012 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:19:54.426862 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b8b1cd-dde3-4872-b7f3-6c2bdac553eb admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:54.438561 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5627e7b3-91ef-46f1-8cde-8a04342e6b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:54.439650 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5627e7b3-91ef-46f1-8cde-8a04342e6b6e admin admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:54.445159 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5627e7b3-91ef-46f1-8cde-8a04342e6b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:54.445159 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5627e7b3-91ef-46f1-8cde-8a04342e6b6e admin admin] PATH_INFO: `/v3/projects/1bd73284f4b44ba1867060a82fadf7e2/users/c5da0151e5fb4335bc916b3940c5409a/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:54.460664 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5627e7b3-91ef-46f1-8cde-8a04342e6b6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bd73284f4b44ba1867060a82fadf7e2, user_id=c5da0151e5fb4335bc916b3940c5409a, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:54.462872 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5627e7b3-91ef-46f1-8cde-8a04342e6b6e admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:54.492612 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 651/1305] 10.222.0.36 () {58 vars in 1058 bytes} [Wed Oct 12 00:19:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:19:54.503173 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851adfed-14d3-4602-94ee-3048ef9f4982 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:54.516713 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851adfed-14d3-4602-94ee-3048ef9f4982 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:54.528942 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9857e8-f32a-4e7c-bdb1-165be4ca8a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:54.529584 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9857e8-f32a-4e7c-bdb1-165be4ca8a5c admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:54.529857 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9857e8-f32a-4e7c-bdb1-165be4ca8a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:54.530128 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9857e8-f32a-4e7c-bdb1-165be4ca8a5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:54.531391 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9857e8-f32a-4e7c-bdb1-165be4ca8a5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:54.533201 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9857e8-f32a-4e7c-bdb1-165be4ca8a5c admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:54.550292 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 652/1306] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:19:54 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:19:54.556297 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95eb80c-cc4e-43da-8e48-2185749249a8 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:54.556596 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95eb80c-cc4e-43da-8e48-2185749249a8 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:54.556878 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95eb80c-cc4e-43da-8e48-2185749249a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:54.569675 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 655/1307] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:19:54 2022] PUT /identity/v3/projects/1bd73284f4b44ba1867060a82fadf7e2/users/c5da0151e5fb4335bc916b3940c5409a/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:19:54.570728 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5627e7b3-91ef-46f1-8cde-8a04342e6b6e admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:54.592172 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-b95eb80c-cc4e-43da-8e48-2185749249a8 None None] MFA Rules not processed for user `ab89c78210e246c68e33d23bf97a093f`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:19:54.594015 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-b95eb80c-cc4e-43da-8e48-2185749249a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:19:54.676796 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5627e7b3-91ef-46f1-8cde-8a04342e6b6e admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:54.686189 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9f8dd4-371b-45ab-adc7-2c4742855ef2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a515b75aa3944912ad2364ffd7924f40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:54.687047 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f8dd4-371b-45ab-adc7-2c4742855ef2 service nova] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:54.687896 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f8dd4-371b-45ab-adc7-2c4742855ef2 service nova] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:54.688432 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f8dd4-371b-45ab-adc7-2c4742855ef2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:54.743949 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 653/1308] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:19:54 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:19:54.745096 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95eb80c-cc4e-43da-8e48-2185749249a8 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:54.816714 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:19:54.820773 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9f8dd4-371b-45ab-adc7-2c4742855ef2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:54.824779 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9f8dd4-371b-45ab-adc7-2c4742855ef2 service nova] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:54.844015 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95eb80c-cc4e-43da-8e48-2185749249a8 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:54.848988 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1a52f4-a01d-460e-b966-b3d6992ffa7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:54.851015 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1a52f4-a01d-460e-b966-b3d6992ffa7a admin admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:54.851015 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1a52f4-a01d-460e-b966-b3d6992ffa7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:54.851015 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1a52f4-a01d-460e-b966-b3d6992ffa7a admin admin] PATH_INFO: `/v3/projects/199062130e6f4f6b8f289940f42d1c86/users/42fe4f5c05fd41df84387f1e2886d7a7/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:54.854098 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 656/1309] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:19:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:19:54.855279 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9f8dd4-371b-45ab-adc7-2c4742855ef2 service nova] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:54.866363 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9f8dd4-371b-45ab-adc7-2c4742855ef2 service nova] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:54.873182 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78a37c6-93e3-4b42-bc14-0341d33b080b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:54.873886 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a37c6-93e3-4b42-bc14-0341d33b080b admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:54.874234 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a37c6-93e3-4b42-bc14-0341d33b080b admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:54.874704 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a37c6-93e3-4b42-bc14-0341d33b080b admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:54.876417 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78a37c6-93e3-4b42-bc14-0341d33b080b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:54.878852 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78a37c6-93e3-4b42-bc14-0341d33b080b admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:54.892562 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 657/1310] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:19:54 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:19:54.897771 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78a37c6-93e3-4b42-bc14-0341d33b080b admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:54.901320 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1a52f4-a01d-460e-b966-b3d6992ffa7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=199062130e6f4f6b8f289940f42d1c86, user_id=42fe4f5c05fd41df84387f1e2886d7a7, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:54.905088 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1a52f4-a01d-460e-b966-b3d6992ffa7a admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:54.984367 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78a37c6-93e3-4b42-bc14-0341d33b080b admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:54.989618 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca5bf5f-c1d3-446a-a0aa-3271854b2e2d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:54.990604 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca5bf5f-c1d3-446a-a0aa-3271854b2e2d service cinder] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:54.991144 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca5bf5f-c1d3-446a-a0aa-3271854b2e2d service cinder] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:54.991508 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca5bf5f-c1d3-446a-a0aa-3271854b2e2d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:55.001392 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 654/1311] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:19:54 2022] PUT /identity/v3/projects/199062130e6f4f6b8f289940f42d1c86/users/42fe4f5c05fd41df84387f1e2886d7a7/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:19:55.002308 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1a52f4-a01d-460e-b966-b3d6992ffa7a admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:55.006371 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca5bf5f-c1d3-446a-a0aa-3271854b2e2d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:55.008576 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca5bf5f-c1d3-446a-a0aa-3271854b2e2d service cinder] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:55.024165 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 658/1312] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:19:54 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:19:55.027071 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca5bf5f-c1d3-446a-a0aa-3271854b2e2d service cinder] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:55.045753 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca5bf5f-c1d3-446a-a0aa-3271854b2e2d service cinder] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:55.130874 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1a52f4-a01d-460e-b966-b3d6992ffa7a admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:55.137618 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8849149-300a-43a6-b0a1-34b9e29f8c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:55.138417 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8849149-300a-43a6-b0a1-34b9e29f8c20 admin admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:55.138699 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8849149-300a-43a6-b0a1-34b9e29f8c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:55.139228 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8849149-300a-43a6-b0a1-34b9e29f8c20 admin admin] PATH_INFO: `/v3/projects/1bd73284f4b44ba1867060a82fadf7e2/users/c5da0151e5fb4335bc916b3940c5409a/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:55.141495 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8849149-300a-43a6-b0a1-34b9e29f8c20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bd73284f4b44ba1867060a82fadf7e2, user_id=c5da0151e5fb4335bc916b3940c5409a, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:55.159617 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:19:55.160944 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8849149-300a-43a6-b0a1-34b9e29f8c20 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:55.196582 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 655/1313] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:19:55 2022] PUT /identity/v3/projects/1bd73284f4b44ba1867060a82fadf7e2/users/c5da0151e5fb4335bc916b3940c5409a/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:19:55.198466 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce14e0-48c1-42e3-b813-30ee023dde06 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:55.198982 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce14e0-48c1-42e3-b813-30ee023dde06 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:55.199554 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce14e0-48c1-42e3-b813-30ee023dde06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:55.205014 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061379bf-4b3d-4176-88c0-3a2a69b970a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:55.213477 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061379bf-4b3d-4176-88c0-3a2a69b970a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:55.213477 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061379bf-4b3d-4176-88c0-3a2a69b970a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:55.213477 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061379bf-4b3d-4176-88c0-3a2a69b970a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:55.213477 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061379bf-4b3d-4176-88c0-3a2a69b970a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:55.213477 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061379bf-4b3d-4176-88c0-3a2a69b970a5 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:55.236230 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 659/1314] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:19:55 2022] GET /identity/v3/roles => generated 1420 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:19:55.240508 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061379bf-4b3d-4176-88c0-3a2a69b970a5 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:55.251761 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-17ce14e0-48c1-42e3-b813-30ee023dde06 None None] MFA Rules not processed for user `c5da0151e5fb4335bc916b3940c5409a`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:19:55.253444 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-17ce14e0-48c1-42e3-b813-30ee023dde06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:19:55.283805 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061379bf-4b3d-4176-88c0-3a2a69b970a5 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:55.439293 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f55bf46-8134-4c89-8fd9-2976febfffac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:55.440496 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f55bf46-8134-4c89-8fd9-2976febfffac admin admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:55.440496 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f55bf46-8134-4c89-8fd9-2976febfffac admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:55.440986 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f55bf46-8134-4c89-8fd9-2976febfffac admin admin] PATH_INFO: `/v3/projects/199062130e6f4f6b8f289940f42d1c86/users/42fe4f5c05fd41df84387f1e2886d7a7/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:55.445570 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f55bf46-8134-4c89-8fd9-2976febfffac admin admin] RBAC: Authorizing `identity:create_grant(project_id=199062130e6f4f6b8f289940f42d1c86, user_id=42fe4f5c05fd41df84387f1e2886d7a7, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:55.451782 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 656/1315] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:19:55 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:19:55.456629 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f55bf46-8134-4c89-8fd9-2976febfffac admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:55.463776 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ce14e0-48c1-42e3-b813-30ee023dde06 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:55.493337 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 660/1316] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:19:55 2022] PUT /identity/v3/projects/199062130e6f4f6b8f289940f42d1c86/users/42fe4f5c05fd41df84387f1e2886d7a7/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:19:55.496975 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8455ef05-b523-41df-bb0b-10b66f451f67 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:55.497266 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8455ef05-b523-41df-bb0b-10b66f451f67 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:55.497542 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8455ef05-b523-41df-bb0b-10b66f451f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:55.575525 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-8455ef05-b523-41df-bb0b-10b66f451f67 None None] MFA Rules not processed for user `42fe4f5c05fd41df84387f1e2886d7a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:19:55.576913 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-8455ef05-b523-41df-bb0b-10b66f451f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:19:55.602611 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ce14e0-48c1-42e3-b813-30ee023dde06 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:55.705316 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b1b778-017f-4a8c-8083-e795c4dc0246 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:55.705316 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b1b778-017f-4a8c-8083-e795c4dc0246 service cinder] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:55.705316 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b1b778-017f-4a8c-8083-e795c4dc0246 service cinder] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:55.705316 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b1b778-017f-4a8c-8083-e795c4dc0246 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:55.713016 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 661/1317] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:19:55 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:19:55.789580 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da852efd-47d3-4a94-a0ba-91a6ace22973 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:55.790191 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da852efd-47d3-4a94-a0ba-91a6ace22973 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:55.790618 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da852efd-47d3-4a94-a0ba-91a6ace22973 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:55.792396 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 662/1318] 10.222.0.36 () {54 vars in 961 bytes} [Wed Oct 12 00:19:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 00:19:55.800517 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed39566d-8cf3-4c2c-9e02-e0f17aa64973 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:55.801238 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed39566d-8cf3-4c2c-9e02-e0f17aa64973 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:55.801664 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed39566d-8cf3-4c2c-9e02-e0f17aa64973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:55.834089 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b1b778-017f-4a8c-8083-e795c4dc0246 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:55.835667 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b1b778-017f-4a8c-8083-e795c4dc0246 service cinder] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:55.841647 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-ed39566d-8cf3-4c2c-9e02-e0f17aa64973 None None] MFA Rules not processed for user `a515b75aa3944912ad2364ffd7924f40`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:19:55.849417 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 657/1319] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:19:55 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:19:55.849417 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b1b778-017f-4a8c-8083-e795c4dc0246 service cinder] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:55.852932 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-ed39566d-8cf3-4c2c-9e02-e0f17aa64973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:19:55.858845 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b1b778-017f-4a8c-8083-e795c4dc0246 service cinder] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:55.863217 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daa5876-8c05-48f6-8a63-615b3346daf6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:55.864447 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daa5876-8c05-48f6-8a63-615b3346daf6 service cinder] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:55.864903 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daa5876-8c05-48f6-8a63-615b3346daf6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:55.865330 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daa5876-8c05-48f6-8a63-615b3346daf6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:55.876697 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daa5876-8c05-48f6-8a63-615b3346daf6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:55.881472 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daa5876-8c05-48f6-8a63-615b3346daf6 service cinder] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:55.907553 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 658/1320] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:19:55 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:19:56.016247 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 663/1321] 10.222.0.36 () {58 vars in 1058 bytes} [Wed Oct 12 00:19:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 00:19:56.027621 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e60b78d-6f51-4256-9611-2241e3097383 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:56.027912 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e60b78d-6f51-4256-9611-2241e3097383 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:56.028223 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e60b78d-6f51-4256-9611-2241e3097383 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:56.029748 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 659/1322] 10.222.0.36 () {54 vars in 912 bytes} [Wed Oct 12 00:19:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:19:56.037551 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c17a792-a2fa-495a-a0d2-304fcdb3e2d2 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:56.038586 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c17a792-a2fa-495a-a0d2-304fcdb3e2d2 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:56.039676 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c17a792-a2fa-495a-a0d2-304fcdb3e2d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:56.079271 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-9c17a792-a2fa-495a-a0d2-304fcdb3e2d2 None None] MFA Rules not processed for user `a515b75aa3944912ad2364ffd7924f40`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:19:56.085704 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-9c17a792-a2fa-495a-a0d2-304fcdb3e2d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:19:56.100910 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 664/1323] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Oct 12 00:19:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:19:56.106458 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:19:56.156670 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e60b78d-6f51-4256-9611-2241e3097383 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:56.182270 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e60b78d-6f51-4256-9611-2241e3097383 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:56.188910 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4a6cdd-852f-424f-bba2-d2cff19b975f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:56.189792 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a6cdd-852f-424f-bba2-d2cff19b975f service cinder] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:56.190213 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a6cdd-852f-424f-bba2-d2cff19b975f service cinder] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:56.190878 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a6cdd-852f-424f-bba2-d2cff19b975f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:56.225427 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4a6cdd-852f-424f-bba2-d2cff19b975f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:56.236539 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4a6cdd-852f-424f-bba2-d2cff19b975f service cinder] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:56.267998 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 660/1324] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:19:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:19:56.574792 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c17a792-a2fa-495a-a0d2-304fcdb3e2d2 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:56.576468 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32217dc9-944a-4f2d-afa8-b52328acf6aa None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:56.576468 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32217dc9-944a-4f2d-afa8-b52328acf6aa None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:56.576468 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32217dc9-944a-4f2d-afa8-b52328acf6aa None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:56.576468 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 661/1325] 10.222.0.36 () {54 vars in 912 bytes} [Wed Oct 12 00:19:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 00:19:56.581807 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:19:56.588674 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1eacb6-4659-4b5a-87d3-35daae810f89 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:56.588674 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1eacb6-4659-4b5a-87d3-35daae810f89 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:56.588893 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1eacb6-4659-4b5a-87d3-35daae810f89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:56.633555 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-9a1eacb6-4659-4b5a-87d3-35daae810f89 None None] MFA Rules not processed for user `a515b75aa3944912ad2364ffd7924f40`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:19:56.636588 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-9a1eacb6-4659-4b5a-87d3-35daae810f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:19:56.652675 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 662/1326] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Oct 12 00:19:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Oct 12 00:19:56.659062 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c17a792-a2fa-495a-a0d2-304fcdb3e2d2 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:56.665512 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde01456-6447-4faa-982b-13be231a950a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:56.666817 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde01456-6447-4faa-982b-13be231a950a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:56.667175 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde01456-6447-4faa-982b-13be231a950a admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:56.667410 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde01456-6447-4faa-982b-13be231a950a admin admin] PATH_INFO: `/v3/users/0d0276e935f247a9955fe576224821b4` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:56.669016 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde01456-6447-4faa-982b-13be231a950a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d0276e935f247a9955fe576224821b4)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:56.670971 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde01456-6447-4faa-982b-13be231a950a admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:56.754671 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1eacb6-4659-4b5a-87d3-35daae810f89 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:56.765393 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1eacb6-4659-4b5a-87d3-35daae810f89 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:56.769737 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:19:56.776607 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4739fe-aa7c-4700-9145-f2e70709091e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:56.777395 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4739fe-aa7c-4700-9145-f2e70709091e service cinder] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:56.777980 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4739fe-aa7c-4700-9145-f2e70709091e service cinder] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:56.777980 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4739fe-aa7c-4700-9145-f2e70709091e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:56.791186 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4739fe-aa7c-4700-9145-f2e70709091e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:56.793147 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4739fe-aa7c-4700-9145-f2e70709091e service cinder] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:56.823066 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 663/1327] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:19:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 00:19:56.914675 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-cde01456-6447-4faa-982b-13be231a950a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d0276e935f247a9955fe576224821b4'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:19:56.915696 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-cde01456-6447-4faa-982b-13be231a950a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d0276e935f247a9955fe576224821b4'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:19:56.943190 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-cde01456-6447-4faa-982b-13be231a950a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d0276e935f247a9955fe576224821b4'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:19:57.018713 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-cde01456-6447-4faa-982b-13be231a950a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d0276e935f247a9955fe576224821b4'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:19:57.019326 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-cde01456-6447-4faa-982b-13be231a950a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d0276e935f247a9955fe576224821b4'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:19:57.045938 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 665/1328] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:19:56 2022] DELETE /identity/v3/users/0d0276e935f247a9955fe576224821b4 => generated 0 bytes in 567 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:19:57.048010 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff8198-3a56-42ee-ac9c-92b19f1c1e15 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:57.048297 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff8198-3a56-42ee-ac9c-92b19f1c1e15 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:57.048565 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff8198-3a56-42ee-ac9c-92b19f1c1e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:57.094403 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde01456-6447-4faa-982b-13be231a950a admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:57.098760 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-cde01456-6447-4faa-982b-13be231a950a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:19:57.119626 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-44ff8198-3a56-42ee-ac9c-92b19f1c1e15 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:19:57.122956 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-44ff8198-3a56-42ee-ac9c-92b19f1c1e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:19:57.248393 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 664/1329] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:19:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:19:57.249558 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ff8198-3a56-42ee-ac9c-92b19f1c1e15 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:57.251172 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-44ff8198-3a56-42ee-ac9c-92b19f1c1e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:19:57.269984 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ff8198-3a56-42ee-ac9c-92b19f1c1e15 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:57.278637 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885294c3-0b3d-458b-add1-2365f4299399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:57.279674 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885294c3-0b3d-458b-add1-2365f4299399 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:57.280179 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885294c3-0b3d-458b-add1-2365f4299399 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:57.281253 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885294c3-0b3d-458b-add1-2365f4299399 admin admin] PATH_INFO: `/v3/users/fb2ff6db932541aaaa90bbec30091aa0` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:57.283962 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885294c3-0b3d-458b-add1-2365f4299399 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb2ff6db932541aaaa90bbec30091aa0)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:57.285422 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: WARNING py.warnings [None req-885294c3-0b3d-458b-add1-2365f4299399 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 12 00:19:57.285422 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: warnings.warn(msg) Oct 12 00:19:57.285422 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: Oct 12 00:19:57.287103 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885294c3-0b3d-458b-add1-2365f4299399 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:57.303273 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde01456-6447-4faa-982b-13be231a950a admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:57.309453 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939a2c1c-d9e8-4f51-8d57-6b7ad1415fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:57.310790 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939a2c1c-d9e8-4f51-8d57-6b7ad1415fe2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:57.311223 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939a2c1c-d9e8-4f51-8d57-6b7ad1415fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:57.311668 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939a2c1c-d9e8-4f51-8d57-6b7ad1415fe2 admin admin] PATH_INFO: `/v3/users/d1bb3b1ab7ac4abca069396a138a61e5` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:57.313074 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939a2c1c-d9e8-4f51-8d57-6b7ad1415fe2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1bb3b1ab7ac4abca069396a138a61e5)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:57.315255 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939a2c1c-d9e8-4f51-8d57-6b7ad1415fe2 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:57.584318 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-885294c3-0b3d-458b-add1-2365f4299399 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb2ff6db932541aaaa90bbec30091aa0'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:19:57.586205 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-885294c3-0b3d-458b-add1-2365f4299399 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb2ff6db932541aaaa90bbec30091aa0'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:19:57.586631 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-939a2c1c-d9e8-4f51-8d57-6b7ad1415fe2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1bb3b1ab7ac4abca069396a138a61e5'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:19:57.588167 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-939a2c1c-d9e8-4f51-8d57-6b7ad1415fe2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1bb3b1ab7ac4abca069396a138a61e5'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:19:57.619141 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-885294c3-0b3d-458b-add1-2365f4299399 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb2ff6db932541aaaa90bbec30091aa0'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:19:57.628536 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-939a2c1c-d9e8-4f51-8d57-6b7ad1415fe2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1bb3b1ab7ac4abca069396a138a61e5'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:19:57.707553 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-939a2c1c-d9e8-4f51-8d57-6b7ad1415fe2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1bb3b1ab7ac4abca069396a138a61e5'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:19:57.710247 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-939a2c1c-d9e8-4f51-8d57-6b7ad1415fe2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1bb3b1ab7ac4abca069396a138a61e5'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:19:57.734568 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 666/1330] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:19:57 2022] DELETE /identity/v3/users/d1bb3b1ab7ac4abca069396a138a61e5 => generated 0 bytes in 656 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:19:57.735915 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939a2c1c-d9e8-4f51-8d57-6b7ad1415fe2 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:57.737649 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-939a2c1c-d9e8-4f51-8d57-6b7ad1415fe2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:19:57.767173 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-885294c3-0b3d-458b-add1-2365f4299399 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb2ff6db932541aaaa90bbec30091aa0'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:19:57.768128 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-885294c3-0b3d-458b-add1-2365f4299399 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb2ff6db932541aaaa90bbec30091aa0'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:19:57.805002 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 665/1331] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:19:57 2022] DELETE /identity/v3/users/fb2ff6db932541aaaa90bbec30091aa0 => generated 0 bytes in 553 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:19:57.805002 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885294c3-0b3d-458b-add1-2365f4299399 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:57.805002 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-885294c3-0b3d-458b-add1-2365f4299399 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:19:57.846191 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939a2c1c-d9e8-4f51-8d57-6b7ad1415fe2 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:57.913677 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885294c3-0b3d-458b-add1-2365f4299399 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:57.920634 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af05054-cabe-44dc-b3eb-04c64b0017b2 tempest-VolumeQuotaClassesTest-498296518 tempest-VolumeQuotaClassesTest-498296518-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42fe4f5c05fd41df84387f1e2886d7a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '199062130e6f4f6b8f289940f42d1c86', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:57.922205 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af05054-cabe-44dc-b3eb-04c64b0017b2 tempest-VolumeQuotaClassesTest-498296518 tempest-VolumeQuotaClassesTest-498296518-project-admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:57.922864 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af05054-cabe-44dc-b3eb-04c64b0017b2 tempest-VolumeQuotaClassesTest-498296518 tempest-VolumeQuotaClassesTest-498296518-project-admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:57.923539 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af05054-cabe-44dc-b3eb-04c64b0017b2 tempest-VolumeQuotaClassesTest-498296518 tempest-VolumeQuotaClassesTest-498296518-project-admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:57.924606 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af05054-cabe-44dc-b3eb-04c64b0017b2 tempest-VolumeQuotaClassesTest-498296518 tempest-VolumeQuotaClassesTest-498296518-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:19:57.926054 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af05054-cabe-44dc-b3eb-04c64b0017b2 tempest-VolumeQuotaClassesTest-498296518 tempest-VolumeQuotaClassesTest-498296518-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:57.928451 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af05054-cabe-44dc-b3eb-04c64b0017b2 tempest-VolumeQuotaClassesTest-498296518 tempest-VolumeQuotaClassesTest-498296518-project-admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:57.947317 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 666/1332] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Oct 12 00:19:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:19:57.953262 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac3cbd9-051b-4362-a62e-c373e93822d0 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:57.953483 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac3cbd9-051b-4362-a62e-c373e93822d0 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:57.953709 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac3cbd9-051b-4362-a62e-c373e93822d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:57.980266 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c469890-fdeb-4104-a718-7defeb986eb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:57.980973 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c469890-fdeb-4104-a718-7defeb986eb8 service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:57.981250 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c469890-fdeb-4104-a718-7defeb986eb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:57.982347 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c469890-fdeb-4104-a718-7defeb986eb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:57.987432 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-7c469890-fdeb-4104-a718-7defeb986eb8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:19:58.113349 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c469890-fdeb-4104-a718-7defeb986eb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:58.114843 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c469890-fdeb-4104-a718-7defeb986eb8 service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:58.143180 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:19:58.147113 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 667/1333] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:19:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:19:58.150714 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-aac3cbd9-051b-4362-a62e-c373e93822d0 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:19:58.150869 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb0b54e-f3d6-4e17-a0fe-2c6137e77698 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:58.151090 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb0b54e-f3d6-4e17-a0fe-2c6137e77698 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:58.151397 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb0b54e-f3d6-4e17-a0fe-2c6137e77698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:58.152690 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-aac3cbd9-051b-4362-a62e-c373e93822d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:19:58.167493 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 667/1334] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:19:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:19:58.168614 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac3cbd9-051b-4362-a62e-c373e93822d0 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:58.170690 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-aac3cbd9-051b-4362-a62e-c373e93822d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:19:58.195497 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac3cbd9-051b-4362-a62e-c373e93822d0 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:58.207234 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed4db77-a8fa-4327-8b40-e472da077313 tempest-VolumeQuotaClassesTest-498296518 tempest-VolumeQuotaClassesTest-498296518-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42fe4f5c05fd41df84387f1e2886d7a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '199062130e6f4f6b8f289940f42d1c86', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:58.208201 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed4db77-a8fa-4327-8b40-e472da077313 tempest-VolumeQuotaClassesTest-498296518 tempest-VolumeQuotaClassesTest-498296518-project-admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:58.208569 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed4db77-a8fa-4327-8b40-e472da077313 tempest-VolumeQuotaClassesTest-498296518 tempest-VolumeQuotaClassesTest-498296518-project-admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:58.208896 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed4db77-a8fa-4327-8b40-e472da077313 tempest-VolumeQuotaClassesTest-498296518 tempest-VolumeQuotaClassesTest-498296518-project-admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:58.210162 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed4db77-a8fa-4327-8b40-e472da077313 tempest-VolumeQuotaClassesTest-498296518 tempest-VolumeQuotaClassesTest-498296518-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:58.214683 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-7cb0b54e-f3d6-4e17-a0fe-2c6137e77698 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:19:58.216072 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-7cb0b54e-f3d6-4e17-a0fe-2c6137e77698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:19:58.216423 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed4db77-a8fa-4327-8b40-e472da077313 tempest-VolumeQuotaClassesTest-498296518 tempest-VolumeQuotaClassesTest-498296518-project-admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:58.245906 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 668/1335] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:19:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:19:58.250104 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb0b54e-f3d6-4e17-a0fe-2c6137e77698 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:58.253179 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:19:58.265791 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb0b54e-f3d6-4e17-a0fe-2c6137e77698 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:58.270783 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d36e6b-9b8f-40fc-bca3-fd538b3d93be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:58.273669 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d36e6b-9b8f-40fc-bca3-fd538b3d93be service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:58.274079 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d36e6b-9b8f-40fc-bca3-fd538b3d93be service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:58.274551 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d36e6b-9b8f-40fc-bca3-fd538b3d93be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:58.291470 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d36e6b-9b8f-40fc-bca3-fd538b3d93be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:58.294309 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d36e6b-9b8f-40fc-bca3-fd538b3d93be service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:58.310225 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 669/1336] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:19:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:19:58.311091 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d36e6b-9b8f-40fc-bca3-fd538b3d93be service neutron] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:58.321383 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d36e6b-9b8f-40fc-bca3-fd538b3d93be service neutron] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:58.325451 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 668/1337] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:19:58 2022] POST /identity/v3/projects => generated 353 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:19:58.326402 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed4db77-a8fa-4327-8b40-e472da077313 tempest-VolumeQuotaClassesTest-498296518 tempest-VolumeQuotaClassesTest-498296518-project-admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:58.401361 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7eb154-5ce0-4929-a686-542f3e1fb5a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:58.402015 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7eb154-5ce0-4929-a686-542f3e1fb5a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:58.402336 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7eb154-5ce0-4929-a686-542f3e1fb5a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:58.402962 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7eb154-5ce0-4929-a686-542f3e1fb5a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:58.491733 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7eb154-5ce0-4929-a686-542f3e1fb5a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:58.493216 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7eb154-5ce0-4929-a686-542f3e1fb5a6 service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:58.509645 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 670/1338] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:19:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:19:58.521171 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed4db77-a8fa-4327-8b40-e472da077313 tempest-VolumeQuotaClassesTest-498296518 tempest-VolumeQuotaClassesTest-498296518-project-admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:58.531225 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d05f33-afa3-4908-a536-51e2cd004de8 tempest-VolumeQuotaClassesTest-498296518 tempest-VolumeQuotaClassesTest-498296518-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42fe4f5c05fd41df84387f1e2886d7a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '199062130e6f4f6b8f289940f42d1c86', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:58.535250 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d05f33-afa3-4908-a536-51e2cd004de8 tempest-VolumeQuotaClassesTest-498296518 tempest-VolumeQuotaClassesTest-498296518-project-admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:58.535250 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d05f33-afa3-4908-a536-51e2cd004de8 tempest-VolumeQuotaClassesTest-498296518 tempest-VolumeQuotaClassesTest-498296518-project-admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:58.535250 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d05f33-afa3-4908-a536-51e2cd004de8 tempest-VolumeQuotaClassesTest-498296518 tempest-VolumeQuotaClassesTest-498296518-project-admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:58.535250 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d05f33-afa3-4908-a536-51e2cd004de8 tempest-VolumeQuotaClassesTest-498296518 tempest-VolumeQuotaClassesTest-498296518-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:19:58.535250 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d05f33-afa3-4908-a536-51e2cd004de8 tempest-VolumeQuotaClassesTest-498296518 tempest-VolumeQuotaClassesTest-498296518-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:58.553414 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d05f33-afa3-4908-a536-51e2cd004de8 tempest-VolumeQuotaClassesTest-498296518 tempest-VolumeQuotaClassesTest-498296518-project-admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:58.575294 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 669/1339] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Oct 12 00:19:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:19:58.641601 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7eb154-5ce0-4929-a686-542f3e1fb5a6 service neutron] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:58.664115 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7eb154-5ce0-4929-a686-542f3e1fb5a6 service neutron] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:58.668945 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7e3b1e-fdb1-4e37-872a-1bdbdcc1d615 tempest-VolumeQuotaClassesTest-498296518 tempest-VolumeQuotaClassesTest-498296518-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42fe4f5c05fd41df84387f1e2886d7a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '199062130e6f4f6b8f289940f42d1c86', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:58.673412 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7e3b1e-fdb1-4e37-872a-1bdbdcc1d615 tempest-VolumeQuotaClassesTest-498296518 tempest-VolumeQuotaClassesTest-498296518-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:58.673860 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7e3b1e-fdb1-4e37-872a-1bdbdcc1d615 tempest-VolumeQuotaClassesTest-498296518 tempest-VolumeQuotaClassesTest-498296518-project-admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:58.673976 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7e3b1e-fdb1-4e37-872a-1bdbdcc1d615 tempest-VolumeQuotaClassesTest-498296518 tempest-VolumeQuotaClassesTest-498296518-project-admin] PATH_INFO: `/v3/projects/2fa040f58b054a1483b0a80ea4ab2227` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:58.675239 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7e3b1e-fdb1-4e37-872a-1bdbdcc1d615 tempest-VolumeQuotaClassesTest-498296518 tempest-VolumeQuotaClassesTest-498296518-project-admin] RBAC: Authorizing `identity:delete_project(project_id=2fa040f58b054a1483b0a80ea4ab2227)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:58.676318 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: WARNING py.warnings [None req-fb7e3b1e-fdb1-4e37-872a-1bdbdcc1d615 tempest-VolumeQuotaClassesTest-498296518 tempest-VolumeQuotaClassesTest-498296518-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 12 00:19:58.676318 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: warnings.warn(msg) Oct 12 00:19:58.676318 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: Oct 12 00:19:58.677503 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7e3b1e-fdb1-4e37-872a-1bdbdcc1d615 tempest-VolumeQuotaClassesTest-498296518 tempest-VolumeQuotaClassesTest-498296518-project-admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:19:58.869083 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-fb7e3b1e-fdb1-4e37-872a-1bdbdcc1d615 tempest-VolumeQuotaClassesTest-498296518 tempest-VolumeQuotaClassesTest-498296518-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fa040f58b054a1483b0a80ea4ab2227', 'request_id': 'req-fb7e3b1e-fdb1-4e37-872a-1bdbdcc1d615'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:19:58.898428 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-fb7e3b1e-fdb1-4e37-872a-1bdbdcc1d615 tempest-VolumeQuotaClassesTest-498296518 tempest-VolumeQuotaClassesTest-498296518-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fa040f58b054a1483b0a80ea4ab2227', 'request_id': 'req-fb7e3b1e-fdb1-4e37-872a-1bdbdcc1d615'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:19:58.983961 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-fb7e3b1e-fdb1-4e37-872a-1bdbdcc1d615 tempest-VolumeQuotaClassesTest-498296518 tempest-VolumeQuotaClassesTest-498296518-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fa040f58b054a1483b0a80ea4ab2227', 'request_id': 'req-fb7e3b1e-fdb1-4e37-872a-1bdbdcc1d615'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:19:59.006427 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-fb7e3b1e-fdb1-4e37-872a-1bdbdcc1d615 tempest-VolumeQuotaClassesTest-498296518 tempest-VolumeQuotaClassesTest-498296518-project-admin] The token cache is being invalidate because project 2fa040f58b054a1483b0a80ea4ab2227 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81746) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:19:59.006726 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-fb7e3b1e-fdb1-4e37-872a-1bdbdcc1d615 tempest-VolumeQuotaClassesTest-498296518 tempest-VolumeQuotaClassesTest-498296518-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:19:59.015517 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-fb7e3b1e-fdb1-4e37-872a-1bdbdcc1d615 tempest-VolumeQuotaClassesTest-498296518 tempest-VolumeQuotaClassesTest-498296518-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:19:59.018571 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 671/1340] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:19:58 2022] DELETE /identity/v3/projects/2fa040f58b054a1483b0a80ea4ab2227 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:19:59.728644 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d05f33-afa3-4908-a536-51e2cd004de8 tempest-VolumeQuotaClassesTest-498296518 tempest-VolumeQuotaClassesTest-498296518-project-admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:19:59.730428 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-b2d05f33-afa3-4908-a536-51e2cd004de8 tempest-VolumeQuotaClassesTest-498296518 tempest-VolumeQuotaClassesTest-498296518-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:19:59.764287 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d05f33-afa3-4908-a536-51e2cd004de8 tempest-VolumeQuotaClassesTest-498296518 tempest-VolumeQuotaClassesTest-498296518-project-admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:19:59.785686 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:19:59.881150 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccbeac3-aeff-48d0-93a8-027ebbcb1026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:19:59.882745 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccbeac3-aeff-48d0-93a8-027ebbcb1026 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:19:59.883027 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccbeac3-aeff-48d0-93a8-027ebbcb1026 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:19:59.883296 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccbeac3-aeff-48d0-93a8-027ebbcb1026 admin admin] PATH_INFO: `/v3/projects/a73b565e3ac34231a7df03912dcf74fb` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:19:59.884883 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccbeac3-aeff-48d0-93a8-027ebbcb1026 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a73b565e3ac34231a7df03912dcf74fb)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:19:59.888307 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccbeac3-aeff-48d0-93a8-027ebbcb1026 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:00.022423 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7e3b1e-fdb1-4e37-872a-1bdbdcc1d615 tempest-VolumeQuotaClassesTest-498296518 tempest-VolumeQuotaClassesTest-498296518-project-admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:00.024923 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-fb7e3b1e-fdb1-4e37-872a-1bdbdcc1d615 tempest-VolumeQuotaClassesTest-498296518 tempest-VolumeQuotaClassesTest-498296518-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:00.048555 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7e3b1e-fdb1-4e37-872a-1bdbdcc1d615 tempest-VolumeQuotaClassesTest-498296518 tempest-VolumeQuotaClassesTest-498296518-project-admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:00.063267 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-4ccbeac3-aeff-48d0-93a8-027ebbcb1026 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a73b565e3ac34231a7df03912dcf74fb', 'request_id': 'req-4ccbeac3-aeff-48d0-93a8-027ebbcb1026'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:00.079749 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-4ccbeac3-aeff-48d0-93a8-027ebbcb1026 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a73b565e3ac34231a7df03912dcf74fb', 'request_id': 'req-4ccbeac3-aeff-48d0-93a8-027ebbcb1026'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:00.152863 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accf05ce-1361-4511-bbc0-c752ef160892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:00.154369 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accf05ce-1361-4511-bbc0-c752ef160892 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:00.154927 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accf05ce-1361-4511-bbc0-c752ef160892 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:00.156021 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accf05ce-1361-4511-bbc0-c752ef160892 admin admin] PATH_INFO: `/v3/users/e1bf651b8858485392bd2dcfec738e2c` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:00.157566 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accf05ce-1361-4511-bbc0-c752ef160892 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1bf651b8858485392bd2dcfec738e2c)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:00.160255 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accf05ce-1361-4511-bbc0-c752ef160892 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:00.178842 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-4ccbeac3-aeff-48d0-93a8-027ebbcb1026 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a73b565e3ac34231a7df03912dcf74fb', 'request_id': 'req-4ccbeac3-aeff-48d0-93a8-027ebbcb1026'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:00.208450 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-4ccbeac3-aeff-48d0-93a8-027ebbcb1026 admin admin] The token cache is being invalidate because project a73b565e3ac34231a7df03912dcf74fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81747) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:20:00.210980 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:00.212416 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-4ccbeac3-aeff-48d0-93a8-027ebbcb1026 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:00.215403 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-4ccbeac3-aeff-48d0-93a8-027ebbcb1026 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:00.221505 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 670/1341] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:19:59 2022] DELETE /identity/v3/projects/a73b565e3ac34231a7df03912dcf74fb => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:00.224446 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccbeac3-aeff-48d0-93a8-027ebbcb1026 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:00.226648 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-4ccbeac3-aeff-48d0-93a8-027ebbcb1026 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:00.257685 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccbeac3-aeff-48d0-93a8-027ebbcb1026 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:00.281980 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdb2885-037a-4bdc-900b-798bd9106112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:00.282657 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb2885-037a-4bdc-900b-798bd9106112 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:00.282936 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb2885-037a-4bdc-900b-798bd9106112 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:00.283212 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb2885-037a-4bdc-900b-798bd9106112 admin admin] PATH_INFO: `/v3/users/c5da0151e5fb4335bc916b3940c5409a` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:00.284386 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdb2885-037a-4bdc-900b-798bd9106112 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5da0151e5fb4335bc916b3940c5409a)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:00.286349 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdb2885-037a-4bdc-900b-798bd9106112 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:00.491819 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-accf05ce-1361-4511-bbc0-c752ef160892 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1bf651b8858485392bd2dcfec738e2c'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:00.492782 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-accf05ce-1361-4511-bbc0-c752ef160892 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1bf651b8858485392bd2dcfec738e2c'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:00.534157 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-accf05ce-1361-4511-bbc0-c752ef160892 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1bf651b8858485392bd2dcfec738e2c'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:00.621005 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-accf05ce-1361-4511-bbc0-c752ef160892 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1bf651b8858485392bd2dcfec738e2c'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:00.621695 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-accf05ce-1361-4511-bbc0-c752ef160892 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1bf651b8858485392bd2dcfec738e2c'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:00.633894 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-3cdb2885-037a-4bdc-900b-798bd9106112 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5da0151e5fb4335bc916b3940c5409a'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:00.634622 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-3cdb2885-037a-4bdc-900b-798bd9106112 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5da0151e5fb4335bc916b3940c5409a'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:00.648282 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 672/1342] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:20:00 2022] DELETE /identity/v3/users/e1bf651b8858485392bd2dcfec738e2c => generated 0 bytes in 627 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 00:20:00.652098 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accf05ce-1361-4511-bbc0-c752ef160892 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:00.664227 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-3cdb2885-037a-4bdc-900b-798bd9106112 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5da0151e5fb4335bc916b3940c5409a'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:00.664552 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-accf05ce-1361-4511-bbc0-c752ef160892 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:00.750753 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-3cdb2885-037a-4bdc-900b-798bd9106112 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5da0151e5fb4335bc916b3940c5409a'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:00.751420 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-3cdb2885-037a-4bdc-900b-798bd9106112 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5da0151e5fb4335bc916b3940c5409a'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:00.784435 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 671/1343] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:20:00 2022] DELETE /identity/v3/users/c5da0151e5fb4335bc916b3940c5409a => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:00.786822 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169977fe-5fc9-4588-8f8a-2cc019dd90b1 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:00.787197 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169977fe-5fc9-4588-8f8a-2cc019dd90b1 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:00.787406 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169977fe-5fc9-4588-8f8a-2cc019dd90b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:00.836513 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-169977fe-5fc9-4588-8f8a-2cc019dd90b1 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:00.838303 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accf05ce-1361-4511-bbc0-c752ef160892 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:00.842431 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-169977fe-5fc9-4588-8f8a-2cc019dd90b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:00.856242 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:01.033757 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e946455b-b9ed-46f5-8975-501c4d256e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:01.034503 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e946455b-b9ed-46f5-8975-501c4d256e78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:01.034692 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e946455b-b9ed-46f5-8975-501c4d256e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:01.034965 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e946455b-b9ed-46f5-8975-501c4d256e78 admin admin] PATH_INFO: `/v3/projects/5175428cacd149c792aff29b84c5631a` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:01.037611 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e946455b-b9ed-46f5-8975-501c4d256e78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5175428cacd149c792aff29b84c5631a)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:01.048217 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 672/1344] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:01.055866 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169977fe-5fc9-4588-8f8a-2cc019dd90b1 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:01.059192 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e946455b-b9ed-46f5-8975-501c4d256e78 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:01.067130 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-169977fe-5fc9-4588-8f8a-2cc019dd90b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:01.109900 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169977fe-5fc9-4588-8f8a-2cc019dd90b1 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:01.133519 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4b955d-1334-483a-9c92-b9764887eea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:01.134187 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b955d-1334-483a-9c92-b9764887eea4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:01.134466 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b955d-1334-483a-9c92-b9764887eea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:01.134739 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b955d-1334-483a-9c92-b9764887eea4 admin admin] PATH_INFO: `/v3/projects/532f356fff2a4e8db81955a7071bdb9b` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:01.140736 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4b955d-1334-483a-9c92-b9764887eea4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=532f356fff2a4e8db81955a7071bdb9b)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:01.145110 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4b955d-1334-483a-9c92-b9764887eea4 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:01.195872 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:01.402328 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-ba4b955d-1334-483a-9c92-b9764887eea4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '532f356fff2a4e8db81955a7071bdb9b', 'request_id': 'req-ba4b955d-1334-483a-9c92-b9764887eea4'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:01.430242 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-ba4b955d-1334-483a-9c92-b9764887eea4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '532f356fff2a4e8db81955a7071bdb9b', 'request_id': 'req-ba4b955d-1334-483a-9c92-b9764887eea4'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:01.553128 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-ba4b955d-1334-483a-9c92-b9764887eea4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '532f356fff2a4e8db81955a7071bdb9b', 'request_id': 'req-ba4b955d-1334-483a-9c92-b9764887eea4'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:01.566587 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-ba4b955d-1334-483a-9c92-b9764887eea4 admin admin] The token cache is being invalidate because project 532f356fff2a4e8db81955a7071bdb9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81747) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:20:01.566788 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-ba4b955d-1334-483a-9c92-b9764887eea4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:01.567537 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-ba4b955d-1334-483a-9c92-b9764887eea4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:01.572812 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 673/1345] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:20:01 2022] DELETE /identity/v3/projects/532f356fff2a4e8db81955a7071bdb9b => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:01.573864 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4b955d-1334-483a-9c92-b9764887eea4 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:01.575588 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-ba4b955d-1334-483a-9c92-b9764887eea4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:01.628327 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-e946455b-b9ed-46f5-8975-501c4d256e78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5175428cacd149c792aff29b84c5631a', 'request_id': 'req-e946455b-b9ed-46f5-8975-501c4d256e78'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:01.648779 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4b955d-1334-483a-9c92-b9764887eea4 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:01.649644 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-e946455b-b9ed-46f5-8975-501c4d256e78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5175428cacd149c792aff29b84c5631a', 'request_id': 'req-e946455b-b9ed-46f5-8975-501c4d256e78'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:01.732400 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57701b06-afe7-4639-9f83-6fb86d7fbe33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:01.735125 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57701b06-afe7-4639-9f83-6fb86d7fbe33 service neutron] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:01.735546 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57701b06-afe7-4639-9f83-6fb86d7fbe33 service neutron] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:01.735824 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57701b06-afe7-4639-9f83-6fb86d7fbe33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:01.738255 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-e946455b-b9ed-46f5-8975-501c4d256e78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5175428cacd149c792aff29b84c5631a', 'request_id': 'req-e946455b-b9ed-46f5-8975-501c4d256e78'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:01.753124 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-57701b06-afe7-4639-9f83-6fb86d7fbe33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:01.754526 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-e946455b-b9ed-46f5-8975-501c4d256e78 admin admin] The token cache is being invalidate because project 5175428cacd149c792aff29b84c5631a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81746) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:20:01.755087 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-e946455b-b9ed-46f5-8975-501c4d256e78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:01.756430 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-e946455b-b9ed-46f5-8975-501c4d256e78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:01.758576 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 673/1346] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:20:00 2022] DELETE /identity/v3/projects/5175428cacd149c792aff29b84c5631a => generated 0 bytes in 1110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:01.779199 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e260ae85-9499-4184-bdf6-bf68de238ac1 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:01.779690 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e260ae85-9499-4184-bdf6-bf68de238ac1 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:01.780108 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e260ae85-9499-4184-bdf6-bf68de238ac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:01.824058 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-e260ae85-9499-4184-bdf6-bf68de238ac1 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:01.825480 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-e260ae85-9499-4184-bdf6-bf68de238ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:01.913685 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57701b06-afe7-4639-9f83-6fb86d7fbe33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:01.915160 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57701b06-afe7-4639-9f83-6fb86d7fbe33 service neutron] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:01.926397 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 674/1347] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:01.927472 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e260ae85-9499-4184-bdf6-bf68de238ac1 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:01.929694 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-e260ae85-9499-4184-bdf6-bf68de238ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:01.944883 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e260ae85-9499-4184-bdf6-bf68de238ac1 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:01.945084 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 674/1348] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:20:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:20:01.948440 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1bcb0f-9ba3-4aeb-b10e-25a909554ffe None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:01.949039 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1bcb0f-9ba3-4aeb-b10e-25a909554ffe None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:01.950433 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1bcb0f-9ba3-4aeb-b10e-25a909554ffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:01.951777 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a09587e-5346-413b-8183-ca954c1ffb6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:01.952470 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a09587e-5346-413b-8183-ca954c1ffb6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:01.952750 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a09587e-5346-413b-8183-ca954c1ffb6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:01.953023 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a09587e-5346-413b-8183-ca954c1ffb6b admin admin] PATH_INFO: `/v3/users/ab89c78210e246c68e33d23bf97a093f` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:01.954652 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a09587e-5346-413b-8183-ca954c1ffb6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab89c78210e246c68e33d23bf97a093f)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:01.956719 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a09587e-5346-413b-8183-ca954c1ffb6b admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:02.011751 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-0a1bcb0f-9ba3-4aeb-b10e-25a909554ffe None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:02.011751 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-0a1bcb0f-9ba3-4aeb-b10e-25a909554ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:02.033390 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 675/1349] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:02.037450 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1bcb0f-9ba3-4aeb-b10e-25a909554ffe None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:02.058145 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1bcb0f-9ba3-4aeb-b10e-25a909554ffe None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:02.063387 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b43d1d-c1ac-40f9-8b59-e023f61b1e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:02.067968 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b43d1d-c1ac-40f9-8b59-e023f61b1e4c admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:02.067968 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b43d1d-c1ac-40f9-8b59-e023f61b1e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:02.067968 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b43d1d-c1ac-40f9-8b59-e023f61b1e4c admin admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:02.067968 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b43d1d-c1ac-40f9-8b59-e023f61b1e4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:20:02.067968 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b43d1d-c1ac-40f9-8b59-e023f61b1e4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:02.067968 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b43d1d-c1ac-40f9-8b59-e023f61b1e4c admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:02.097157 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 676/1350] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Oct 12 00:20:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 00:20:02.098619 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b43d1d-c1ac-40f9-8b59-e023f61b1e4c admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:02.118428 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b43d1d-c1ac-40f9-8b59-e023f61b1e4c admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:02.132264 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cae04dd-36c4-42ba-b635-1e817faee6a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:02.132911 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cae04dd-36c4-42ba-b635-1e817faee6a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:02.133176 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cae04dd-36c4-42ba-b635-1e817faee6a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:02.133448 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cae04dd-36c4-42ba-b635-1e817faee6a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:02.134093 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cae04dd-36c4-42ba-b635-1e817faee6a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:02.136064 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cae04dd-36c4-42ba-b635-1e817faee6a3 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:02.210207 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-1a09587e-5346-413b-8183-ca954c1ffb6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab89c78210e246c68e33d23bf97a093f'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:02.212933 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-1a09587e-5346-413b-8183-ca954c1ffb6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab89c78210e246c68e33d23bf97a093f'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:02.274212 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 677/1351] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:20:02 2022] POST /identity/v3/projects => generated 365 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:20:02.284829 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cae04dd-36c4-42ba-b635-1e817faee6a3 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:02.286645 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-7cae04dd-36c4-42ba-b635-1e817faee6a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:02.317907 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-1a09587e-5346-413b-8183-ca954c1ffb6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab89c78210e246c68e33d23bf97a093f'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:02.403836 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-1a09587e-5346-413b-8183-ca954c1ffb6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab89c78210e246c68e33d23bf97a093f'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:02.404468 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-1a09587e-5346-413b-8183-ca954c1ffb6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab89c78210e246c68e33d23bf97a093f'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:02.447598 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 675/1352] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:20:01 2022] DELETE /identity/v3/users/ab89c78210e246c68e33d23bf97a093f => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:02.481883 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cae04dd-36c4-42ba-b635-1e817faee6a3 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:02.564166 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:02.603949 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893de5d0-b8ce-4b31-8759-838859a25308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:02.604596 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893de5d0-b8ce-4b31-8759-838859a25308 admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:02.604859 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893de5d0-b8ce-4b31-8759-838859a25308 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:02.605128 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893de5d0-b8ce-4b31-8759-838859a25308 admin admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:02.606258 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893de5d0-b8ce-4b31-8759-838859a25308 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:02.612928 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893de5d0-b8ce-4b31-8759-838859a25308 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:02.746667 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:02.834530 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 678/1353] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:20:02 2022] POST /identity/v3/users => generated 283 bytes in 550 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 00:20:02.839418 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a09587e-5346-413b-8183-ca954c1ffb6b admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:02.856297 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a09587e-5346-413b-8183-ca954c1ffb6b admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:02.875838 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc06501-a45b-4bce-b009-c6ce3efa67ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:02.876604 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc06501-a45b-4bce-b009-c6ce3efa67ae admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:02.876977 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc06501-a45b-4bce-b009-c6ce3efa67ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:02.877347 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc06501-a45b-4bce-b009-c6ce3efa67ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:02.878124 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc06501-a45b-4bce-b009-c6ce3efa67ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:02.881084 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc06501-a45b-4bce-b009-c6ce3efa67ae admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:02.907988 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 676/1354] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:20:02 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:20:02.910306 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893de5d0-b8ce-4b31-8759-838859a25308 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:02.923017 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893de5d0-b8ce-4b31-8759-838859a25308 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:02.940520 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32522b0-83f4-4409-8964-04b1dcb8c1ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:02.941161 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32522b0-83f4-4409-8964-04b1dcb8c1ed admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:02.941432 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32522b0-83f4-4409-8964-04b1dcb8c1ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:02.941941 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32522b0-83f4-4409-8964-04b1dcb8c1ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:02.942960 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32522b0-83f4-4409-8964-04b1dcb8c1ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:02.945618 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32522b0-83f4-4409-8964-04b1dcb8c1ed admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:02.970918 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc06501-a45b-4bce-b009-c6ce3efa67ae admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:02.981827 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc06501-a45b-4bce-b009-c6ce3efa67ae admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:02.982511 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 679/1355] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:20:02 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:20:02.988455 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a8eb58-7ab6-4006-b7f0-40cc75b6b75e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:02.989454 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a8eb58-7ab6-4006-b7f0-40cc75b6b75e admin admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:02.993163 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a8eb58-7ab6-4006-b7f0-40cc75b6b75e admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:02.993163 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a8eb58-7ab6-4006-b7f0-40cc75b6b75e admin admin] PATH_INFO: `/v3/projects/a14e5c00a0ff4333b5931205f19bfe91/users/c24aa3f7f951483d991e59c93f01ce49/roles/ef689f96442945e8a7bc8cbfee13d909` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:03.012872 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a8eb58-7ab6-4006-b7f0-40cc75b6b75e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a14e5c00a0ff4333b5931205f19bfe91, user_id=c24aa3f7f951483d991e59c93f01ce49, role_id=ef689f96442945e8a7bc8cbfee13d909)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:03.022279 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32522b0-83f4-4409-8964-04b1dcb8c1ed admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:03.024664 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-c32522b0-83f4-4409-8964-04b1dcb8c1ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:03.030429 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a8eb58-7ab6-4006-b7f0-40cc75b6b75e admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:03.047009 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32522b0-83f4-4409-8964-04b1dcb8c1ed admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:03.061108 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce32e88a-6633-497b-81de-9290f6593171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:03.062664 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce32e88a-6633-497b-81de-9290f6593171 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:03.062949 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce32e88a-6633-497b-81de-9290f6593171 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:03.063271 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce32e88a-6633-497b-81de-9290f6593171 admin admin] PATH_INFO: `/v3/projects/1bd73284f4b44ba1867060a82fadf7e2` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:03.065932 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce32e88a-6633-497b-81de-9290f6593171 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bd73284f4b44ba1867060a82fadf7e2)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:03.073685 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce32e88a-6633-497b-81de-9290f6593171 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:03.167461 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 677/1356] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:20:02 2022] PUT /identity/v3/projects/a14e5c00a0ff4333b5931205f19bfe91/users/c24aa3f7f951483d991e59c93f01ce49/roles/ef689f96442945e8a7bc8cbfee13d909 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:03.169070 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da6a8f-bdfe-46b9-9025-f29ff594b43b None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:03.169351 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da6a8f-bdfe-46b9-9025-f29ff594b43b None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:03.179874 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da6a8f-bdfe-46b9-9025-f29ff594b43b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:03.220529 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-ce32e88a-6633-497b-81de-9290f6593171 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bd73284f4b44ba1867060a82fadf7e2', 'request_id': 'req-ce32e88a-6633-497b-81de-9290f6593171'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:03.236639 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-ce32e88a-6633-497b-81de-9290f6593171 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bd73284f4b44ba1867060a82fadf7e2', 'request_id': 'req-ce32e88a-6633-497b-81de-9290f6593171'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:03.269196 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-13da6a8f-bdfe-46b9-9025-f29ff594b43b None None] MFA Rules not processed for user `c24aa3f7f951483d991e59c93f01ce49`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:03.274813 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-13da6a8f-bdfe-46b9-9025-f29ff594b43b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:03.335610 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-ce32e88a-6633-497b-81de-9290f6593171 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bd73284f4b44ba1867060a82fadf7e2', 'request_id': 'req-ce32e88a-6633-497b-81de-9290f6593171'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:03.358905 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-ce32e88a-6633-497b-81de-9290f6593171 admin admin] The token cache is being invalidate because project 1bd73284f4b44ba1867060a82fadf7e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81747) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:20:03.361620 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-ce32e88a-6633-497b-81de-9290f6593171 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:03.364950 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-ce32e88a-6633-497b-81de-9290f6593171 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:03.373405 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 680/1357] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:20:03 2022] DELETE /identity/v3/projects/1bd73284f4b44ba1867060a82fadf7e2 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:03.386856 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b89a7e-9bf3-45a4-9898-7a2b6a8f4225 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:03.398189 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b89a7e-9bf3-45a4-9898-7a2b6a8f4225 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:03.398796 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b89a7e-9bf3-45a4-9898-7a2b6a8f4225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:03.458710 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 678/1358] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:03 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:03.463700 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13da6a8f-bdfe-46b9-9025-f29ff594b43b None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:03.469629 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-13da6a8f-bdfe-46b9-9025-f29ff594b43b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:03.473250 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-18b89a7e-9bf3-45a4-9898-7a2b6a8f4225 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:03.475951 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-18b89a7e-9bf3-45a4-9898-7a2b6a8f4225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:03.615659 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13da6a8f-bdfe-46b9-9025-f29ff594b43b None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:03.627634 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 681/1359] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:03.633995 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c69bf-d08c-472f-ac47-cdf4fc7a4bac None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:03.634288 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c69bf-d08c-472f-ac47-cdf4fc7a4bac None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:03.634528 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c69bf-d08c-472f-ac47-cdf4fc7a4bac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:03.636365 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecb9371-55cf-457c-8379-09e51de90786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:03.637239 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecb9371-55cf-457c-8379-09e51de90786 admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:03.637709 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecb9371-55cf-457c-8379-09e51de90786 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:03.638160 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecb9371-55cf-457c-8379-09e51de90786 admin admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:03.639446 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecb9371-55cf-457c-8379-09e51de90786 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:03.651837 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecb9371-55cf-457c-8379-09e51de90786 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:03.670568 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-d51c69bf-d08c-472f-ac47-cdf4fc7a4bac None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:03.673021 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-d51c69bf-d08c-472f-ac47-cdf4fc7a4bac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:03.688379 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 682/1360] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 00:20:03.692883 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51c69bf-d08c-472f-ac47-cdf4fc7a4bac None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:03.707674 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51c69bf-d08c-472f-ac47-cdf4fc7a4bac None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:03.721143 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f83ba0-c39c-455a-96db-956140e934d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:03.722785 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f83ba0-c39c-455a-96db-956140e934d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:03.724137 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f83ba0-c39c-455a-96db-956140e934d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:03.724714 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f83ba0-c39c-455a-96db-956140e934d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:03.726230 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f83ba0-c39c-455a-96db-956140e934d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:20:03.726723 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f83ba0-c39c-455a-96db-956140e934d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:03.729028 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f83ba0-c39c-455a-96db-956140e934d4 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:03.746376 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 683/1361] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Oct 12 00:20:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:20:03.751025 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f83ba0-c39c-455a-96db-956140e934d4 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:03.772665 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 679/1362] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:20:03 2022] POST /identity/v3/projects => generated 361 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:20:03.776386 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecb9371-55cf-457c-8379-09e51de90786 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:03.784799 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f83ba0-c39c-455a-96db-956140e934d4 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:03.957977 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecb9371-55cf-457c-8379-09e51de90786 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:03.958913 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c94b81f-c600-4e4b-aa1f-d9e97404a3bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:03.960413 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c94b81f-c600-4e4b-aa1f-d9e97404a3bc admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:03.961671 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c94b81f-c600-4e4b-aa1f-d9e97404a3bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:03.962461 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c94b81f-c600-4e4b-aa1f-d9e97404a3bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:03.965415 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c94b81f-c600-4e4b-aa1f-d9e97404a3bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:03.974196 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c94b81f-c600-4e4b-aa1f-d9e97404a3bc admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:03.975107 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d34b321-7952-485a-bd09-82696905da1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:03.977073 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d34b321-7952-485a-bd09-82696905da1a admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:03.977568 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d34b321-7952-485a-bd09-82696905da1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:03.979117 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d34b321-7952-485a-bd09-82696905da1a admin admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:03.980587 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d34b321-7952-485a-bd09-82696905da1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:03.982731 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d34b321-7952-485a-bd09-82696905da1a admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:04.098135 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 684/1363] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:20:03 2022] POST /identity/v3/projects => generated 373 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:20:04.101943 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c94b81f-c600-4e4b-aa1f-d9e97404a3bc admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:04.126397 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 680/1364] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:20:03 2022] POST /identity/v3/users => generated 287 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:20:04.130726 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d34b321-7952-485a-bd09-82696905da1a admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:04.147776 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d34b321-7952-485a-bd09-82696905da1a admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:04.269500 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2a15af-4050-4fa7-a028-d25c3c8e338a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:04.272399 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c94b81f-c600-4e4b-aa1f-d9e97404a3bc admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:04.273620 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a15af-4050-4fa7-a028-d25c3c8e338a admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:04.274133 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a15af-4050-4fa7-a028-d25c3c8e338a admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:04.274711 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a15af-4050-4fa7-a028-d25c3c8e338a admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:04.275936 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2a15af-4050-4fa7-a028-d25c3c8e338a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:04.277890 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2a15af-4050-4fa7-a028-d25c3c8e338a admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:04.288230 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bc475c-f007-4429-9882-d9ac4c648f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:04.288871 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bc475c-f007-4429-9882-d9ac4c648f3f admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:04.289150 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bc475c-f007-4429-9882-d9ac4c648f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:04.289590 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bc475c-f007-4429-9882-d9ac4c648f3f admin admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:04.290603 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bc475c-f007-4429-9882-d9ac4c648f3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:04.292444 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 681/1365] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:20:04 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:20:04.296905 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2a15af-4050-4fa7-a028-d25c3c8e338a admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:04.300702 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bc475c-f007-4429-9882-d9ac4c648f3f admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:04.310961 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2a15af-4050-4fa7-a028-d25c3c8e338a admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:04.318079 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7e66d2-d936-41aa-be13-fa1c991cfe89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:04.319200 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7e66d2-d936-41aa-be13-fa1c991cfe89 admin admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:04.319738 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7e66d2-d936-41aa-be13-fa1c991cfe89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:04.320902 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7e66d2-d936-41aa-be13-fa1c991cfe89 admin admin] PATH_INFO: `/v3/projects/ba0e3f466bee4bbf8a92a23a2ccdb80a/users/099caafb51b742028bb978fb6fb43c09/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:04.356925 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7e66d2-d936-41aa-be13-fa1c991cfe89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba0e3f466bee4bbf8a92a23a2ccdb80a, user_id=099caafb51b742028bb978fb6fb43c09, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:04.361636 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7e66d2-d936-41aa-be13-fa1c991cfe89 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:04.408823 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 685/1366] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:20:04 2022] POST /identity/v3/users => generated 287 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:20:04.409807 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bc475c-f007-4429-9882-d9ac4c648f3f admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:04.432770 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bc475c-f007-4429-9882-d9ac4c648f3f admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:04.436950 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 682/1367] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:20:04 2022] PUT /identity/v3/projects/ba0e3f466bee4bbf8a92a23a2ccdb80a/users/099caafb51b742028bb978fb6fb43c09/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 00:20:04.441693 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7e66d2-d936-41aa-be13-fa1c991cfe89 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:04.496817 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:04.595940 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:04.596830 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7e66d2-d936-41aa-be13-fa1c991cfe89 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:04.601339 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a419c39-4470-4a19-aa42-59347c668806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:04.602976 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a419c39-4470-4a19-aa42-59347c668806 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:04.604336 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a419c39-4470-4a19-aa42-59347c668806 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:04.604932 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a419c39-4470-4a19-aa42-59347c668806 admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:04.605376 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a419c39-4470-4a19-aa42-59347c668806 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:04.609773 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a419c39-4470-4a19-aa42-59347c668806 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:04.613066 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82801fb1-7157-4ed4-8c7d-e604d41bc345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:04.613762 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82801fb1-7157-4ed4-8c7d-e604d41bc345 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:04.614039 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82801fb1-7157-4ed4-8c7d-e604d41bc345 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:04.615677 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82801fb1-7157-4ed4-8c7d-e604d41bc345 admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:04.617201 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82801fb1-7157-4ed4-8c7d-e604d41bc345 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:04.620307 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82801fb1-7157-4ed4-8c7d-e604d41bc345 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:04.642390 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 686/1368] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:20:04 2022] GET /identity/v3/roles => generated 1420 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:20:04.643843 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a419c39-4470-4a19-aa42-59347c668806 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:04.656818 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 683/1369] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:20:04 2022] GET /identity/v3/roles => generated 1420 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 00:20:04.657734 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82801fb1-7157-4ed4-8c7d-e604d41bc345 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:04.674788 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a419c39-4470-4a19-aa42-59347c668806 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:04.681248 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82801fb1-7157-4ed4-8c7d-e604d41bc345 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:04.688241 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9969a6-eff4-48d7-91ed-e5eb84f061b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:04.690195 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9969a6-eff4-48d7-91ed-e5eb84f061b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:04.690481 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9969a6-eff4-48d7-91ed-e5eb84f061b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:04.691156 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9969a6-eff4-48d7-91ed-e5eb84f061b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:04.691421 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9969a6-eff4-48d7-91ed-e5eb84f061b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:04.693860 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260f5e64-6870-43db-97f0-837073fb8b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:04.695839 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f5e64-6870-43db-97f0-837073fb8b53 admin admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:04.696117 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f5e64-6870-43db-97f0-837073fb8b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:04.697127 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9969a6-eff4-48d7-91ed-e5eb84f061b9 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:04.697449 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f5e64-6870-43db-97f0-837073fb8b53 admin admin] PATH_INFO: `/v3/projects/ba0e3f466bee4bbf8a92a23a2ccdb80a/users/099caafb51b742028bb978fb6fb43c09/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:04.704605 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260f5e64-6870-43db-97f0-837073fb8b53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba0e3f466bee4bbf8a92a23a2ccdb80a, user_id=099caafb51b742028bb978fb6fb43c09, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:04.707299 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260f5e64-6870-43db-97f0-837073fb8b53 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:04.717908 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 687/1370] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:20:04 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:20:04.719036 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9969a6-eff4-48d7-91ed-e5eb84f061b9 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:04.737118 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 684/1371] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:20:04 2022] PUT /identity/v3/projects/ba0e3f466bee4bbf8a92a23a2ccdb80a/users/099caafb51b742028bb978fb6fb43c09/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:04.747509 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdabc83-7164-44de-b6cd-c4518d198927 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:04.747798 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdabc83-7164-44de-b6cd-c4518d198927 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:04.748068 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdabc83-7164-44de-b6cd-c4518d198927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:04.757807 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9969a6-eff4-48d7-91ed-e5eb84f061b9 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:04.807086 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-0fdabc83-7164-44de-b6cd-c4518d198927 None None] MFA Rules not processed for user `099caafb51b742028bb978fb6fb43c09`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:04.818972 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-0fdabc83-7164-44de-b6cd-c4518d198927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:04.941770 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5fdad6-ebc3-49f1-bb85-765d459fa3c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:04.942513 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5fdad6-ebc3-49f1-bb85-765d459fa3c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:04.942799 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5fdad6-ebc3-49f1-bb85-765d459fa3c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:04.943073 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5fdad6-ebc3-49f1-bb85-765d459fa3c4 admin admin] PATH_INFO: `/v3/projects/f73ba417c10941a79c47a83f17bb5f19/users/f34ed0e065504d608ae920fdbac701d9/roles/ef689f96442945e8a7bc8cbfee13d909` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:04.962304 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 685/1372] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:04 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 00:20:04.976689 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdabc83-7164-44de-b6cd-c4518d198927 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:04.982199 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-0fdabc83-7164-44de-b6cd-c4518d198927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:04.986458 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5fdad6-ebc3-49f1-bb85-765d459fa3c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f73ba417c10941a79c47a83f17bb5f19, user_id=f34ed0e065504d608ae920fdbac701d9, role_id=ef689f96442945e8a7bc8cbfee13d909)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:04.989591 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5fdad6-ebc3-49f1-bb85-765d459fa3c4 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:05.080151 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdabc83-7164-44de-b6cd-c4518d198927 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:05.084284 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a244ad6-5924-45ae-bd44-e882cf144c33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:05.085099 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a244ad6-5924-45ae-bd44-e882cf144c33 service cinder] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:05.085467 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a244ad6-5924-45ae-bd44-e882cf144c33 service cinder] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:05.085861 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a244ad6-5924-45ae-bd44-e882cf144c33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:05.107558 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a244ad6-5924-45ae-bd44-e882cf144c33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:05.115888 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a244ad6-5924-45ae-bd44-e882cf144c33 service cinder] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:05.140567 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 688/1373] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:20:04 2022] PUT /identity/v3/projects/f73ba417c10941a79c47a83f17bb5f19/users/f34ed0e065504d608ae920fdbac701d9/roles/ef689f96442945e8a7bc8cbfee13d909 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:05.143102 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5fdad6-ebc3-49f1-bb85-765d459fa3c4 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:05.145105 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-5e5fdad6-ebc3-49f1-bb85-765d459fa3c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:05.148336 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 686/1374] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:20:04 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:20:05.154725 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90b44a8-a538-4d94-988d-f6f8185278d5 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:05.158778 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90b44a8-a538-4d94-988d-f6f8185278d5 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:05.158778 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90b44a8-a538-4d94-988d-f6f8185278d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:05.253685 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-b90b44a8-a538-4d94-988d-f6f8185278d5 None None] MFA Rules not processed for user `f34ed0e065504d608ae920fdbac701d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:05.254938 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-b90b44a8-a538-4d94-988d-f6f8185278d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:05.320420 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5fdad6-ebc3-49f1-bb85-765d459fa3c4 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:05.340440 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db257e62-f9db-4e26-b1d1-a9277fa49393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:05.341183 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db257e62-f9db-4e26-b1d1-a9277fa49393 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:05.341553 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db257e62-f9db-4e26-b1d1-a9277fa49393 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:05.341831 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db257e62-f9db-4e26-b1d1-a9277fa49393 admin admin] PATH_INFO: `/v3/projects/a963d5b895ca4aa3abd1406893e4b572` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:05.343886 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db257e62-f9db-4e26-b1d1-a9277fa49393 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a963d5b895ca4aa3abd1406893e4b572)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:05.348266 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db257e62-f9db-4e26-b1d1-a9277fa49393 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:05.412023 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 687/1375] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:05.417132 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90b44a8-a538-4d94-988d-f6f8185278d5 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:05.431952 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90b44a8-a538-4d94-988d-f6f8185278d5 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:05.444300 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2431970-40e1-462c-be8f-2858e6411ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:05.444300 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2431970-40e1-462c-be8f-2858e6411ed5 admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:05.444300 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2431970-40e1-462c-be8f-2858e6411ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:05.444300 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2431970-40e1-462c-be8f-2858e6411ed5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:05.444755 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2431970-40e1-462c-be8f-2858e6411ed5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:05.446969 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2431970-40e1-462c-be8f-2858e6411ed5 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:05.530829 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-db257e62-f9db-4e26-b1d1-a9277fa49393 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a963d5b895ca4aa3abd1406893e4b572', 'request_id': 'req-db257e62-f9db-4e26-b1d1-a9277fa49393'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:05.538615 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-db257e62-f9db-4e26-b1d1-a9277fa49393 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a963d5b895ca4aa3abd1406893e4b572', 'request_id': 'req-db257e62-f9db-4e26-b1d1-a9277fa49393'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:05.583102 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 688/1376] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:20:05 2022] POST /identity/v3/projects => generated 373 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:20:05.584241 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2431970-40e1-462c-be8f-2858e6411ed5 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:05.642128 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-db257e62-f9db-4e26-b1d1-a9277fa49393 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a963d5b895ca4aa3abd1406893e4b572', 'request_id': 'req-db257e62-f9db-4e26-b1d1-a9277fa49393'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:05.663211 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-db257e62-f9db-4e26-b1d1-a9277fa49393 admin admin] The token cache is being invalidate because project a963d5b895ca4aa3abd1406893e4b572 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81747) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:20:05.663540 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-db257e62-f9db-4e26-b1d1-a9277fa49393 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:05.664343 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-db257e62-f9db-4e26-b1d1-a9277fa49393 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:05.670812 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 689/1377] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:20:05 2022] DELETE /identity/v3/projects/a963d5b895ca4aa3abd1406893e4b572 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:05.674217 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db257e62-f9db-4e26-b1d1-a9277fa49393 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:05.675754 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-db257e62-f9db-4e26-b1d1-a9277fa49393 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:05.712133 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:05.717695 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2431970-40e1-462c-be8f-2858e6411ed5 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:05.726704 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e6ad42-5ea0-4a51-a122-4f670df1db13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:05.727585 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e6ad42-5ea0-4a51-a122-4f670df1db13 service cinder] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:05.728230 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e6ad42-5ea0-4a51-a122-4f670df1db13 service cinder] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:05.728230 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e6ad42-5ea0-4a51-a122-4f670df1db13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:05.736210 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-54e6ad42-5ea0-4a51-a122-4f670df1db13 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:05.884769 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db257e62-f9db-4e26-b1d1-a9277fa49393 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:05.900341 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e6ad42-5ea0-4a51-a122-4f670df1db13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:05.901821 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e6ad42-5ea0-4a51-a122-4f670df1db13 service cinder] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:05.937835 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 689/1378] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:20:05 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:20:05.938840 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e6ad42-5ea0-4a51-a122-4f670df1db13 service cinder] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:05.948551 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-54e6ad42-5ea0-4a51-a122-4f670df1db13 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:05.971530 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748fcf8f-3689-4572-bd04-80b98b84c7b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:05.973930 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748fcf8f-3689-4572-bd04-80b98b84c7b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:05.973930 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748fcf8f-3689-4572-bd04-80b98b84c7b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:05.973930 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748fcf8f-3689-4572-bd04-80b98b84c7b2 admin admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:05.973930 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748fcf8f-3689-4572-bd04-80b98b84c7b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:05.976596 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748fcf8f-3689-4572-bd04-80b98b84c7b2 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:06.002278 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:06.066849 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e6ad42-5ea0-4a51-a122-4f670df1db13 service cinder] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:06.072373 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502a70eb-7c6a-4ae2-b9fc-4d945592da48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:06.073090 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a70eb-7c6a-4ae2-b9fc-4d945592da48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:06.073500 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a70eb-7c6a-4ae2-b9fc-4d945592da48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:06.073875 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a70eb-7c6a-4ae2-b9fc-4d945592da48 admin admin] PATH_INFO: `/v3/projects/fee2b5f3f77f44bf9d929d752aa351c3` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:06.075288 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502a70eb-7c6a-4ae2-b9fc-4d945592da48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fee2b5f3f77f44bf9d929d752aa351c3)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:06.078021 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502a70eb-7c6a-4ae2-b9fc-4d945592da48 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:06.091643 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 690/1379] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:20:05 2022] POST /identity/v3/users => generated 293 bytes in 418 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:20:06.091643 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748fcf8f-3689-4572-bd04-80b98b84c7b2 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:06.092262 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-748fcf8f-3689-4572-bd04-80b98b84c7b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:06.118856 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748fcf8f-3689-4572-bd04-80b98b84c7b2 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:06.135801 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fee5329-cd06-4f34-bf7c-301bc7c95f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:06.136178 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee5329-cd06-4f34-bf7c-301bc7c95f74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:06.136641 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee5329-cd06-4f34-bf7c-301bc7c95f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:06.137379 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee5329-cd06-4f34-bf7c-301bc7c95f74 admin admin] PATH_INFO: `/v3/users/42fe4f5c05fd41df84387f1e2886d7a7` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:06.141119 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fee5329-cd06-4f34-bf7c-301bc7c95f74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42fe4f5c05fd41df84387f1e2886d7a7)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:06.146524 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fee5329-cd06-4f34-bf7c-301bc7c95f74 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:06.244938 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-502a70eb-7c6a-4ae2-b9fc-4d945592da48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fee2b5f3f77f44bf9d929d752aa351c3', 'request_id': 'req-502a70eb-7c6a-4ae2-b9fc-4d945592da48'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:06.256524 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-502a70eb-7c6a-4ae2-b9fc-4d945592da48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fee2b5f3f77f44bf9d929d752aa351c3', 'request_id': 'req-502a70eb-7c6a-4ae2-b9fc-4d945592da48'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:06.385816 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-502a70eb-7c6a-4ae2-b9fc-4d945592da48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fee2b5f3f77f44bf9d929d752aa351c3', 'request_id': 'req-502a70eb-7c6a-4ae2-b9fc-4d945592da48'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:06.400277 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-502a70eb-7c6a-4ae2-b9fc-4d945592da48 admin admin] The token cache is being invalidate because project fee2b5f3f77f44bf9d929d752aa351c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81746) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:20:06.400604 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-502a70eb-7c6a-4ae2-b9fc-4d945592da48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:06.401273 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-502a70eb-7c6a-4ae2-b9fc-4d945592da48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:06.406384 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 690/1380] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:20:05 2022] DELETE /identity/v3/projects/fee2b5f3f77f44bf9d929d752aa351c3 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:06.406384 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502a70eb-7c6a-4ae2-b9fc-4d945592da48 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:06.406384 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-502a70eb-7c6a-4ae2-b9fc-4d945592da48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:06.458677 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-2fee5329-cd06-4f34-bf7c-301bc7c95f74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42fe4f5c05fd41df84387f1e2886d7a7'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:06.459596 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-2fee5329-cd06-4f34-bf7c-301bc7c95f74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42fe4f5c05fd41df84387f1e2886d7a7'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:06.512278 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-2fee5329-cd06-4f34-bf7c-301bc7c95f74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42fe4f5c05fd41df84387f1e2886d7a7'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:06.595348 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-2fee5329-cd06-4f34-bf7c-301bc7c95f74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42fe4f5c05fd41df84387f1e2886d7a7'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:06.596052 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-2fee5329-cd06-4f34-bf7c-301bc7c95f74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42fe4f5c05fd41df84387f1e2886d7a7'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:06.610138 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502a70eb-7c6a-4ae2-b9fc-4d945592da48 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:06.625661 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 691/1381] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:20:06 2022] DELETE /identity/v3/users/42fe4f5c05fd41df84387f1e2886d7a7 => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:06.626990 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3627f19-4b41-4799-b265-5bf86430791a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:06.628119 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3627f19-4b41-4799-b265-5bf86430791a admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:06.628119 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3627f19-4b41-4799-b265-5bf86430791a admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:06.633212 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e717274-f97c-4ab0-9247-6d00aa3acc17 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:06.633479 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3627f19-4b41-4799-b265-5bf86430791a admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:06.633557 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e717274-f97c-4ab0-9247-6d00aa3acc17 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:06.633557 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e717274-f97c-4ab0-9247-6d00aa3acc17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:06.633805 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3627f19-4b41-4799-b265-5bf86430791a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:06.633805 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3627f19-4b41-4799-b265-5bf86430791a admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:06.675808 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 691/1382] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:20:06 2022] GET /identity/v3/roles => generated 1420 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:20:06.676898 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3627f19-4b41-4799-b265-5bf86430791a admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:06.678638 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-c3627f19-4b41-4799-b265-5bf86430791a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:06.690783 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-6e717274-f97c-4ab0-9247-6d00aa3acc17 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:06.695632 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-6e717274-f97c-4ab0-9247-6d00aa3acc17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:06.704502 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3627f19-4b41-4799-b265-5bf86430791a admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:06.868745 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffe08d1-4ba4-42a4-87a3-8a1d57ef03cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:06.871195 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffe08d1-4ba4-42a4-87a3-8a1d57ef03cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:06.871706 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffe08d1-4ba4-42a4-87a3-8a1d57ef03cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:06.872161 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffe08d1-4ba4-42a4-87a3-8a1d57ef03cc admin admin] PATH_INFO: `/v3/projects/60c8fb2d6b1a4f6696e901317fd99ae7/users/7f946504b5574228bea5d7c88504e6f7/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:06.890290 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 692/1383] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:06.891933 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31565b8-33e9-44ce-9dc4-fbbd7d0ef892 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:06.894053 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31565b8-33e9-44ce-9dc4-fbbd7d0ef892 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:06.894546 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31565b8-33e9-44ce-9dc4-fbbd7d0ef892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:06.901437 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffe08d1-4ba4-42a4-87a3-8a1d57ef03cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=60c8fb2d6b1a4f6696e901317fd99ae7, user_id=7f946504b5574228bea5d7c88504e6f7, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:06.911648 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffe08d1-4ba4-42a4-87a3-8a1d57ef03cc admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:06.966239 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-a31565b8-33e9-44ce-9dc4-fbbd7d0ef892 None None] MFA Rules not processed for user `bca53aca7b134df1949deb10858f14d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:06.970564 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-a31565b8-33e9-44ce-9dc4-fbbd7d0ef892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:06.987160 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 692/1384] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:20:06 2022] PUT /identity/v3/projects/60c8fb2d6b1a4f6696e901317fd99ae7/users/7f946504b5574228bea5d7c88504e6f7/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:06.988618 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeba037-eee9-4077-a779-9f28927feb8b None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:06.988908 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeba037-eee9-4077-a779-9f28927feb8b None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:06.989231 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeba037-eee9-4077-a779-9f28927feb8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:07.046387 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-2aeba037-eee9-4077-a779-9f28927feb8b None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:07.052076 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-2aeba037-eee9-4077-a779-9f28927feb8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:07.123536 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:07.129131 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 693/1385] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:07.148205 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31565b8-33e9-44ce-9dc4-fbbd7d0ef892 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:07.168147 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31565b8-33e9-44ce-9dc4-fbbd7d0ef892 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:07.214536 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 693/1386] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:07.215925 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeba037-eee9-4077-a779-9f28927feb8b None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:07.217618 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-2aeba037-eee9-4077-a779-9f28927feb8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:07.342001 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e93bd1b-80b2-49b4-b0e6-eef804428e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:07.346198 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e93bd1b-80b2-49b4-b0e6-eef804428e25 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:07.346198 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e93bd1b-80b2-49b4-b0e6-eef804428e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:07.346198 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e93bd1b-80b2-49b4-b0e6-eef804428e25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:07.346198 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e93bd1b-80b2-49b4-b0e6-eef804428e25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:07.347573 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e93bd1b-80b2-49b4-b0e6-eef804428e25 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:07.373624 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:07.410772 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 694/1387] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:20:07 2022] GET /identity/v3/roles => generated 1420 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:20:07.412053 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e93bd1b-80b2-49b4-b0e6-eef804428e25 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:07.424765 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e93bd1b-80b2-49b4-b0e6-eef804428e25 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:07.425718 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeba037-eee9-4077-a779-9f28927feb8b None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:07.432173 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25877f8-0593-4452-af6a-c17c53459157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:07.433209 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25877f8-0593-4452-af6a-c17c53459157 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:07.433728 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25877f8-0593-4452-af6a-c17c53459157 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:07.434215 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25877f8-0593-4452-af6a-c17c53459157 admin admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:07.435890 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25877f8-0593-4452-af6a-c17c53459157 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:20:07.436965 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25877f8-0593-4452-af6a-c17c53459157 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:07.439086 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25877f8-0593-4452-af6a-c17c53459157 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:07.447702 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26588bf3-0fe3-4712-8b9e-29b65ed08d29 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a515b75aa3944912ad2364ffd7924f40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:07.450121 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26588bf3-0fe3-4712-8b9e-29b65ed08d29 service nova] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:07.450551 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26588bf3-0fe3-4712-8b9e-29b65ed08d29 service nova] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:07.450766 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26588bf3-0fe3-4712-8b9e-29b65ed08d29 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:07.468047 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 695/1388] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Oct 12 00:20:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:20:07.469752 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25877f8-0593-4452-af6a-c17c53459157 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:07.506213 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25877f8-0593-4452-af6a-c17c53459157 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:07.519003 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1b7c5a-454e-4b87-8c42-7260f29e3c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:07.522280 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1b7c5a-454e-4b87-8c42-7260f29e3c00 admin admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:07.522631 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1b7c5a-454e-4b87-8c42-7260f29e3c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:07.524406 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1b7c5a-454e-4b87-8c42-7260f29e3c00 admin admin] PATH_INFO: `/v3/projects/60c8fb2d6b1a4f6696e901317fd99ae7/users/7f946504b5574228bea5d7c88504e6f7/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:07.528602 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1b7c5a-454e-4b87-8c42-7260f29e3c00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60c8fb2d6b1a4f6696e901317fd99ae7, user_id=7f946504b5574228bea5d7c88504e6f7, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:07.534880 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1b7c5a-454e-4b87-8c42-7260f29e3c00 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:07.568664 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 696/1389] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:20:07 2022] PUT /identity/v3/projects/60c8fb2d6b1a4f6696e901317fd99ae7/users/7f946504b5574228bea5d7c88504e6f7/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:07.569750 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1b7c5a-454e-4b87-8c42-7260f29e3c00 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:07.609650 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26588bf3-0fe3-4712-8b9e-29b65ed08d29 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:07.616827 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26588bf3-0fe3-4712-8b9e-29b65ed08d29 service nova] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:07.654818 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 694/1390] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:20:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:20:07.658981 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c314aed-0df5-427b-9028-705142677a47 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:07.659270 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c314aed-0df5-427b-9028-705142677a47 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:07.659619 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c314aed-0df5-427b-9028-705142677a47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:07.753473 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-5c314aed-0df5-427b-9028-705142677a47 None None] MFA Rules not processed for user `7f946504b5574228bea5d7c88504e6f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:07.763004 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-5c314aed-0df5-427b-9028-705142677a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:07.787527 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1b7c5a-454e-4b87-8c42-7260f29e3c00 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:07.804551 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0e98cf-359a-4f9c-8cb5-11b778cb48c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:07.805401 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0e98cf-359a-4f9c-8cb5-11b778cb48c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:07.809190 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0e98cf-359a-4f9c-8cb5-11b778cb48c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:07.809190 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0e98cf-359a-4f9c-8cb5-11b778cb48c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:07.809190 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0e98cf-359a-4f9c-8cb5-11b778cb48c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:07.809190 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0e98cf-359a-4f9c-8cb5-11b778cb48c1 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:07.893651 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:07.929805 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 697/1391] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:20:07 2022] POST /identity/v3/projects => generated 369 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:20:07.931345 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654ab8b3-e0b6-442d-bb03-74d9d4f2a195 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:07.932178 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654ab8b3-e0b6-442d-bb03-74d9d4f2a195 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:07.932999 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654ab8b3-e0b6-442d-bb03-74d9d4f2a195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:07.968552 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 695/1392] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:07 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:07.968552 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c314aed-0df5-427b-9028-705142677a47 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:08.005581 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c314aed-0df5-427b-9028-705142677a47 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:08.019076 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-654ab8b3-e0b6-442d-bb03-74d9d4f2a195 None None] MFA Rules not processed for user `cdd87151f3af4a3bb1c3815db5d0e8fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:08.022133 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-654ab8b3-e0b6-442d-bb03-74d9d4f2a195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:08.141655 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9d5b3e-d10c-47f7-89ca-c2c90f424eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:08.141813 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:08.142340 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9d5b3e-d10c-47f7-89ca-c2c90f424eb5 admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:08.142626 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9d5b3e-d10c-47f7-89ca-c2c90f424eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:08.142914 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9d5b3e-d10c-47f7-89ca-c2c90f424eb5 admin admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:08.143578 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9d5b3e-d10c-47f7-89ca-c2c90f424eb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:08.146014 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9d5b3e-d10c-47f7-89ca-c2c90f424eb5 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:08.191681 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 698/1393] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:07 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:08.192828 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654ab8b3-e0b6-442d-bb03-74d9d4f2a195 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:08.202402 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-654ab8b3-e0b6-442d-bb03-74d9d4f2a195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:08.289936 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 696/1394] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:20:07 2022] POST /identity/v3/users => generated 285 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:20:08.291324 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d8cab3-3cde-4fe0-aa85-5d7667ae0f70 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:08.291647 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d8cab3-3cde-4fe0-aa85-5d7667ae0f70 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:08.292116 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d8cab3-3cde-4fe0-aa85-5d7667ae0f70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:08.338825 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-f0d8cab3-3cde-4fe0-aa85-5d7667ae0f70 None None] MFA Rules not processed for user `cdd87151f3af4a3bb1c3815db5d0e8fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:08.341270 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-f0d8cab3-3cde-4fe0-aa85-5d7667ae0f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:08.358542 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 697/1395] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:08 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:08.366273 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654ab8b3-e0b6-442d-bb03-74d9d4f2a195 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:08.373059 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d8cab3-3cde-4fe0-aa85-5d7667ae0f70 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:08.378344 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b08a1f5-0d4b-4b47-8914-a1f04e4a2721 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:08.381906 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b08a1f5-0d4b-4b47-8914-a1f04e4a2721 service cinder] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:08.381906 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b08a1f5-0d4b-4b47-8914-a1f04e4a2721 service cinder] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:08.381906 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b08a1f5-0d4b-4b47-8914-a1f04e4a2721 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:08.400421 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d8cab3-3cde-4fe0-aa85-5d7667ae0f70 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:08.411791 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e84c71-30d4-489c-97bb-e26bc4262416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:08.412425 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e84c71-30d4-489c-97bb-e26bc4262416 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:08.412703 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e84c71-30d4-489c-97bb-e26bc4262416 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:08.412973 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e84c71-30d4-489c-97bb-e26bc4262416 admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:08.413780 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e84c71-30d4-489c-97bb-e26bc4262416 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:08.424219 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e84c71-30d4-489c-97bb-e26bc4262416 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:08.443598 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 698/1396] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:20:08 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:20:08.444801 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e84c71-30d4-489c-97bb-e26bc4262416 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:08.448260 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-98e84c71-30d4-489c-97bb-e26bc4262416 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:08.480059 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e84c71-30d4-489c-97bb-e26bc4262416 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:08.495970 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7813535-553d-4475-909a-ae07cff654b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:08.496691 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7813535-553d-4475-909a-ae07cff654b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:08.496976 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7813535-553d-4475-909a-ae07cff654b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:08.497389 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7813535-553d-4475-909a-ae07cff654b0 admin admin] PATH_INFO: `/v3/projects/199062130e6f4f6b8f289940f42d1c86` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:08.498689 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b08a1f5-0d4b-4b47-8914-a1f04e4a2721 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:08.501239 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7813535-553d-4475-909a-ae07cff654b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=199062130e6f4f6b8f289940f42d1c86)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:08.505016 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b08a1f5-0d4b-4b47-8914-a1f04e4a2721 service cinder] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:08.507288 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7813535-553d-4475-909a-ae07cff654b0 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:08.531348 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 699/1397] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:20:08 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:20:08.532596 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b08a1f5-0d4b-4b47-8914-a1f04e4a2721 service cinder] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:08.534331 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-0b08a1f5-0d4b-4b47-8914-a1f04e4a2721 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:08.693591 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-f7813535-553d-4475-909a-ae07cff654b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '199062130e6f4f6b8f289940f42d1c86', 'request_id': 'req-f7813535-553d-4475-909a-ae07cff654b0'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:08.715837 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-f7813535-553d-4475-909a-ae07cff654b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '199062130e6f4f6b8f289940f42d1c86', 'request_id': 'req-f7813535-553d-4475-909a-ae07cff654b0'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:08.722207 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b08a1f5-0d4b-4b47-8914-a1f04e4a2721 service cinder] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:08.832266 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-f7813535-553d-4475-909a-ae07cff654b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '199062130e6f4f6b8f289940f42d1c86', 'request_id': 'req-f7813535-553d-4475-909a-ae07cff654b0'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:08.850218 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-f7813535-553d-4475-909a-ae07cff654b0 admin admin] The token cache is being invalidate because project 199062130e6f4f6b8f289940f42d1c86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81746) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:20:08.850661 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-f7813535-553d-4475-909a-ae07cff654b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:08.852862 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-f7813535-553d-4475-909a-ae07cff654b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:08.857066 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 699/1398] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:20:08 2022] DELETE /identity/v3/projects/199062130e6f4f6b8f289940f42d1c86 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:08.859267 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7813535-553d-4475-909a-ae07cff654b0 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:08.862242 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-f7813535-553d-4475-909a-ae07cff654b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:08.898976 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6d662a-ef5b-4907-bd35-e807f68007e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a515b75aa3944912ad2364ffd7924f40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:08.899387 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d662a-ef5b-4907-bd35-e807f68007e3 service nova] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:08.899667 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d662a-ef5b-4907-bd35-e807f68007e3 service nova] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:08.899942 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d662a-ef5b-4907-bd35-e807f68007e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:08.906280 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-ed6d662a-ef5b-4907-bd35-e807f68007e3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:09.051862 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6d662a-ef5b-4907-bd35-e807f68007e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:09.053707 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6d662a-ef5b-4907-bd35-e807f68007e3 service nova] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:09.085321 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7813535-553d-4475-909a-ae07cff654b0 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:09.105400 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0176e842-09dc-4ab6-a937-6b84c650125c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:09.385061 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0176e842-09dc-4ab6-a937-6b84c650125c admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:09.385061 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0176e842-09dc-4ab6-a937-6b84c650125c admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:09.385061 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0176e842-09dc-4ab6-a937-6b84c650125c admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:09.385061 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0176e842-09dc-4ab6-a937-6b84c650125c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:09.400206 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 700/1399] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:20:08 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 577 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:20:09.400206 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045cb920-36e4-47b9-9b79-56297ca5a94c None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:09.400206 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045cb920-36e4-47b9-9b79-56297ca5a94c None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:09.400206 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045cb920-36e4-47b9-9b79-56297ca5a94c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:09.407136 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0176e842-09dc-4ab6-a937-6b84c650125c admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:09.407136 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 700/1400] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:20:08 2022] GET /identity/v3/roles => generated 1420 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:20:09.407136 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71895188-5447-4c8c-8e76-631b6e6218fb None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:09.407136 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71895188-5447-4c8c-8e76-631b6e6218fb None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:09.407136 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71895188-5447-4c8c-8e76-631b6e6218fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:09.408170 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-045cb920-36e4-47b9-9b79-56297ca5a94c None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:09.408170 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-045cb920-36e4-47b9-9b79-56297ca5a94c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:09.408170 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 701/1401] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:09.408170 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045cb920-36e4-47b9-9b79-56297ca5a94c None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:09.408170 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045cb920-36e4-47b9-9b79-56297ca5a94c None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:09.410654 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-71895188-5447-4c8c-8e76-631b6e6218fb None None] MFA Rules not processed for user `de766cb582ba4c18b1fda9bf4ad43f65`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:09.411204 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3c4250-4b0d-4ee3-821f-baccc96807a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:09.411570 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-71895188-5447-4c8c-8e76-631b6e6218fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:09.411909 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3c4250-4b0d-4ee3-821f-baccc96807a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:09.411909 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3c4250-4b0d-4ee3-821f-baccc96807a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:09.411909 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3c4250-4b0d-4ee3-821f-baccc96807a6 admin admin] PATH_INFO: `/v3/projects/1f81357071c84cbeac45598ec351bed8/users/1fc758a783414aa588ec3604ebed8970/roles/ef689f96442945e8a7bc8cbfee13d909` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:09.411909 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3c4250-4b0d-4ee3-821f-baccc96807a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f81357071c84cbeac45598ec351bed8, user_id=1fc758a783414aa588ec3604ebed8970, role_id=ef689f96442945e8a7bc8cbfee13d909)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:09.411909 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3c4250-4b0d-4ee3-821f-baccc96807a6 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:09.426632 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:09.437430 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 702/1402] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:20:09 2022] PUT /identity/v3/projects/1f81357071c84cbeac45598ec351bed8/users/1fc758a783414aa588ec3604ebed8970/roles/ef689f96442945e8a7bc8cbfee13d909 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:09.438516 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df9a2fd-b4af-431b-a32d-84875e64815a None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:09.438824 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df9a2fd-b4af-431b-a32d-84875e64815a None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:09.439060 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df9a2fd-b4af-431b-a32d-84875e64815a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:09.523066 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-4df9a2fd-b4af-431b-a32d-84875e64815a None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:09.527880 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-4df9a2fd-b4af-431b-a32d-84875e64815a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:09.576084 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 701/1403] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 399 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:09.578137 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c046710e-5eed-4d42-aed4-220d2731fbcf None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:09.579447 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c046710e-5eed-4d42-aed4-220d2731fbcf None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:09.581479 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c046710e-5eed-4d42-aed4-220d2731fbcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:09.650130 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-c046710e-5eed-4d42-aed4-220d2731fbcf None None] MFA Rules not processed for user `1fc758a783414aa588ec3604ebed8970`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:09.651023 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-c046710e-5eed-4d42-aed4-220d2731fbcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:09.781427 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 703/1404] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:09.781427 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df9a2fd-b4af-431b-a32d-84875e64815a None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:09.784308 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-4df9a2fd-b4af-431b-a32d-84875e64815a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:09.876184 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 702/1405] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:09 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:09.877737 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c046710e-5eed-4d42-aed4-220d2731fbcf None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:09.907078 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c046710e-5eed-4d42-aed4-220d2731fbcf None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:09.922129 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd3f48e-a791-4dfb-afff-28e33d28e541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:09.922942 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3f48e-a791-4dfb-afff-28e33d28e541 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:09.923419 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3f48e-a791-4dfb-afff-28e33d28e541 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:09.923822 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3f48e-a791-4dfb-afff-28e33d28e541 admin admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:09.924943 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd3f48e-a791-4dfb-afff-28e33d28e541 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:20:09.925900 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df9a2fd-b4af-431b-a32d-84875e64815a None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:09.926397 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd3f48e-a791-4dfb-afff-28e33d28e541 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:09.929820 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd3f48e-a791-4dfb-afff-28e33d28e541 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:09.959283 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efdf317-72e4-4e52-ba0b-e958f2711716 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a515b75aa3944912ad2364ffd7924f40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:09.962126 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efdf317-72e4-4e52-ba0b-e958f2711716 service nova] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:09.963033 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 703/1406] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Oct 12 00:20:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:20:09.964375 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd3f48e-a791-4dfb-afff-28e33d28e541 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:09.964866 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efdf317-72e4-4e52-ba0b-e958f2711716 service nova] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:09.966754 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efdf317-72e4-4e52-ba0b-e958f2711716 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:09.986357 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd3f48e-a791-4dfb-afff-28e33d28e541 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:10.004773 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810c6c5d-aefb-417e-b67e-a14fc4c56ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:10.005480 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810c6c5d-aefb-417e-b67e-a14fc4c56ad8 admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:10.007648 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810c6c5d-aefb-417e-b67e-a14fc4c56ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:10.007648 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810c6c5d-aefb-417e-b67e-a14fc4c56ad8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:10.007648 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810c6c5d-aefb-417e-b67e-a14fc4c56ad8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:10.015782 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810c6c5d-aefb-417e-b67e-a14fc4c56ad8 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:10.110571 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efdf317-72e4-4e52-ba0b-e958f2711716 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:10.112020 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efdf317-72e4-4e52-ba0b-e958f2711716 service nova] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:10.143041 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 704/1407] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:20:09 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:20:10.147747 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 704/1408] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:20:09 2022] POST /identity/v3/projects => generated 369 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:20:10.148673 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810c6c5d-aefb-417e-b67e-a14fc4c56ad8 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:10.154505 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efdf317-72e4-4e52-ba0b-e958f2711716 service nova] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:10.196738 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efdf317-72e4-4e52-ba0b-e958f2711716 service nova] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:10.301440 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810c6c5d-aefb-417e-b67e-a14fc4c56ad8 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:10.312548 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce381b5-5ae5-4e64-b7bc-db7f7237c574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:10.313546 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce381b5-5ae5-4e64-b7bc-db7f7237c574 admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:10.313887 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce381b5-5ae5-4e64-b7bc-db7f7237c574 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:10.314219 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce381b5-5ae5-4e64-b7bc-db7f7237c574 admin admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:10.315058 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce381b5-5ae5-4e64-b7bc-db7f7237c574 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:10.318104 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982e452b-d4fa-426d-9f8a-ff8b96cdf815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:10.318232 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce381b5-5ae5-4e64-b7bc-db7f7237c574 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:10.318313 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982e452b-d4fa-426d-9f8a-ff8b96cdf815 admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:10.318313 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982e452b-d4fa-426d-9f8a-ff8b96cdf815 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:10.318313 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982e452b-d4fa-426d-9f8a-ff8b96cdf815 admin admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:10.319216 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982e452b-d4fa-426d-9f8a-ff8b96cdf815 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:10.321488 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982e452b-d4fa-426d-9f8a-ff8b96cdf815 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:10.433545 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 705/1409] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:20:10 2022] POST /identity/v3/users => generated 291 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:20:10.433545 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce381b5-5ae5-4e64-b7bc-db7f7237c574 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:10.435778 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-5ce381b5-5ae5-4e64-b7bc-db7f7237c574 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:10.456513 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce381b5-5ae5-4e64-b7bc-db7f7237c574 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:10.470198 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5400e2d-a1b7-496a-b8fb-44db25fd343a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:10.470992 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5400e2d-a1b7-496a-b8fb-44db25fd343a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:10.471346 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5400e2d-a1b7-496a-b8fb-44db25fd343a admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:10.471771 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5400e2d-a1b7-496a-b8fb-44db25fd343a admin admin] PATH_INFO: `/v3/users/f34ed0e065504d608ae920fdbac701d9` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:10.473109 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5400e2d-a1b7-496a-b8fb-44db25fd343a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f34ed0e065504d608ae920fdbac701d9)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:10.475402 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5400e2d-a1b7-496a-b8fb-44db25fd343a admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:10.496718 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 705/1410] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:20:10 2022] POST /identity/v3/projects => generated 365 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:20:10.498528 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a537d-4aac-4adf-bb78-a9b55d890ce0 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:10.498834 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a537d-4aac-4adf-bb78-a9b55d890ce0 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:10.499170 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a537d-4aac-4adf-bb78-a9b55d890ce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:10.600999 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:10.602277 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-4e5a537d-4aac-4adf-bb78-a9b55d890ce0 None None] MFA Rules not processed for user `b4dba90d485945e1821be4667d5a5bc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:10.607002 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-4e5a537d-4aac-4adf-bb78-a9b55d890ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:10.757295 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-c5400e2d-a1b7-496a-b8fb-44db25fd343a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f34ed0e065504d608ae920fdbac701d9'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:10.759193 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 706/1411] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:10 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:10.759703 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-c5400e2d-a1b7-496a-b8fb-44db25fd343a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f34ed0e065504d608ae920fdbac701d9'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:10.760501 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5a537d-4aac-4adf-bb78-a9b55d890ce0 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:10.785890 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-c5400e2d-a1b7-496a-b8fb-44db25fd343a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f34ed0e065504d608ae920fdbac701d9'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:10.844427 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-c5400e2d-a1b7-496a-b8fb-44db25fd343a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f34ed0e065504d608ae920fdbac701d9'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:10.845308 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-c5400e2d-a1b7-496a-b8fb-44db25fd343a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f34ed0e065504d608ae920fdbac701d9'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:10.874344 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 706/1412] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:20:10 2022] DELETE /identity/v3/users/f34ed0e065504d608ae920fdbac701d9 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:10.878340 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5400e2d-a1b7-496a-b8fb-44db25fd343a admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:10.888084 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-c5400e2d-a1b7-496a-b8fb-44db25fd343a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:10.888682 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5a537d-4aac-4adf-bb78-a9b55d890ce0 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:11.009894 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d564d6c-d3f6-4862-9f8a-202406f4b9a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:11.010549 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d564d6c-d3f6-4862-9f8a-202406f4b9a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:11.010824 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d564d6c-d3f6-4862-9f8a-202406f4b9a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:11.011178 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d564d6c-d3f6-4862-9f8a-202406f4b9a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:11.011948 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d564d6c-d3f6-4862-9f8a-202406f4b9a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:11.014811 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d564d6c-d3f6-4862-9f8a-202406f4b9a8 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:11.032635 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5400e2d-a1b7-496a-b8fb-44db25fd343a admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:11.034714 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 707/1413] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:20:10 2022] GET /identity/v3/roles => generated 1420 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:20:11.036411 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e448ca4e-7b8c-4fa0-a427-161a86e3558c None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:11.036856 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e448ca4e-7b8c-4fa0-a427-161a86e3558c None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:11.037681 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e448ca4e-7b8c-4fa0-a427-161a86e3558c None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:11.040599 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 708/1414] 10.222.0.36 () {54 vars in 912 bytes} [Wed Oct 12 00:20:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:20:11.046313 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882cd58e-5217-4b80-9ad0-765104b85b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:11.047177 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882cd58e-5217-4b80-9ad0-765104b85b49 admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:11.047277 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e2aa08-dd0c-46c2-aea6-1d65983a1680 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:11.047513 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e2aa08-dd0c-46c2-aea6-1d65983a1680 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:11.047787 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e2aa08-dd0c-46c2-aea6-1d65983a1680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:11.047966 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882cd58e-5217-4b80-9ad0-765104b85b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:11.048367 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882cd58e-5217-4b80-9ad0-765104b85b49 admin admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:11.049132 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882cd58e-5217-4b80-9ad0-765104b85b49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:11.054236 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882cd58e-5217-4b80-9ad0-765104b85b49 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:11.094485 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-f0e2aa08-dd0c-46c2-aea6-1d65983a1680 None None] MFA Rules not processed for user `b4dba90d485945e1821be4667d5a5bc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:11.096018 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-f0e2aa08-dd0c-46c2-aea6-1d65983a1680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:11.151873 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 707/1415] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:20:10 2022] POST /identity/v3/users => generated 283 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:20:11.153522 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c89f91-d5e5-4b5a-ba6c-f5bc68bf6042 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:11.153945 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c89f91-d5e5-4b5a-ba6c-f5bc68bf6042 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:11.154357 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c89f91-d5e5-4b5a-ba6c-f5bc68bf6042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:11.205592 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 709/1416] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:11.206644 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-46c89f91-d5e5-4b5a-ba6c-f5bc68bf6042 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:11.207720 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e2aa08-dd0c-46c2-aea6-1d65983a1680 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:11.208326 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-46c89f91-d5e5-4b5a-ba6c-f5bc68bf6042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:11.210883 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-f0e2aa08-dd0c-46c2-aea6-1d65983a1680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:11.238703 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 708/1417] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:11.240700 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60baa18-74b7-4ef8-83cd-e5881e1853f7 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:11.241170 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60baa18-74b7-4ef8-83cd-e5881e1853f7 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:11.245058 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60baa18-74b7-4ef8-83cd-e5881e1853f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:11.254406 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e2aa08-dd0c-46c2-aea6-1d65983a1680 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:11.266308 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db7d121-ce2c-4551-9421-bb0af05f49e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:11.267529 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db7d121-ce2c-4551-9421-bb0af05f49e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:11.268978 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db7d121-ce2c-4551-9421-bb0af05f49e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:11.269980 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db7d121-ce2c-4551-9421-bb0af05f49e3 admin admin] PATH_INFO: `/v3/projects/238f3ac862444577beb163e70451deea/users/63a10287986a4c19a07bc674c642b0a2/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:11.305912 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db7d121-ce2c-4551-9421-bb0af05f49e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=238f3ac862444577beb163e70451deea, user_id=63a10287986a4c19a07bc674c642b0a2, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:11.312396 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-f60baa18-74b7-4ef8-83cd-e5881e1853f7 None None] MFA Rules not processed for user `51215dab618141b4991e232724094140`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:11.312952 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db7d121-ce2c-4551-9421-bb0af05f49e3 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:11.314988 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-f60baa18-74b7-4ef8-83cd-e5881e1853f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:11.407798 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 710/1418] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:20:11 2022] PUT /identity/v3/projects/238f3ac862444577beb163e70451deea/users/63a10287986a4c19a07bc674c642b0a2/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:11.408543 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db7d121-ce2c-4551-9421-bb0af05f49e3 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:11.415405 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 709/1419] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Oct 12 00:20:11 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:11.416759 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60baa18-74b7-4ef8-83cd-e5881e1853f7 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:11.421165 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-f60baa18-74b7-4ef8-83cd-e5881e1853f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:11.520680 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60baa18-74b7-4ef8-83cd-e5881e1853f7 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:11.587600 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db7d121-ce2c-4551-9421-bb0af05f49e3 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:11.596531 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce1f416-ae83-4d25-8988-79be871be3e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:11.597180 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce1f416-ae83-4d25-8988-79be871be3e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:11.597468 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce1f416-ae83-4d25-8988-79be871be3e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:11.597793 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce1f416-ae83-4d25-8988-79be871be3e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:11.598423 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce1f416-ae83-4d25-8988-79be871be3e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:11.601925 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce1f416-ae83-4d25-8988-79be871be3e9 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:11.628830 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 711/1420] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:20:11 2022] GET /identity/v3/roles => generated 1420 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:20:11.631954 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce1f416-ae83-4d25-8988-79be871be3e9 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:11.641501 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-4ce1f416-ae83-4d25-8988-79be871be3e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:11.690982 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc204fdd-ff39-4361-9389-d216a225ca2b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a515b75aa3944912ad2364ffd7924f40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:11.692182 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc204fdd-ff39-4361-9389-d216a225ca2b service nova] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:11.692782 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc204fdd-ff39-4361-9389-d216a225ca2b service nova] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:11.702131 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc204fdd-ff39-4361-9389-d216a225ca2b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:11.709516 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-fc204fdd-ff39-4361-9389-d216a225ca2b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:11.758092 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce1f416-ae83-4d25-8988-79be871be3e9 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:11.765812 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:11.767666 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2d642b-c584-4ad9-9a99-f7a20dac791a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:11.769294 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2d642b-c584-4ad9-9a99-f7a20dac791a service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:11.769294 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2d642b-c584-4ad9-9a99-f7a20dac791a service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:11.769294 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2d642b-c584-4ad9-9a99-f7a20dac791a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:11.784487 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2d642b-c584-4ad9-9a99-f7a20dac791a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:11.786065 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2d642b-c584-4ad9-9a99-f7a20dac791a service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:11.800321 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 712/1421] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:20:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:20:11.801432 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2d642b-c584-4ad9-9a99-f7a20dac791a service neutron] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:11.817222 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2d642b-c584-4ad9-9a99-f7a20dac791a service neutron] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:11.825080 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2e365c-c89f-42e7-8a2d-db0c4a2c1562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:11.825749 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2e365c-c89f-42e7-8a2d-db0c4a2c1562 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:11.826086 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2e365c-c89f-42e7-8a2d-db0c4a2c1562 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:11.826363 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2e365c-c89f-42e7-8a2d-db0c4a2c1562 admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:11.827036 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2e365c-c89f-42e7-8a2d-db0c4a2c1562 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:11.828686 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2e365c-c89f-42e7-8a2d-db0c4a2c1562 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:11.831043 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc204fdd-ff39-4361-9389-d216a225ca2b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:11.832776 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc204fdd-ff39-4361-9389-d216a225ca2b service nova] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:11.857556 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 713/1422] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:20:11 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:20:11.858665 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 710/1423] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:20:11 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:20:11.859749 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc204fdd-ff39-4361-9389-d216a225ca2b service nova] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:11.859854 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2e365c-c89f-42e7-8a2d-db0c4a2c1562 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:11.872490 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2e365c-c89f-42e7-8a2d-db0c4a2c1562 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:11.877991 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc204fdd-ff39-4361-9389-d216a225ca2b service nova] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:11.884573 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eb3e22-7bff-477b-9199-69e58b73de7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:11.885240 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eb3e22-7bff-477b-9199-69e58b73de7f admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:11.886522 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eb3e22-7bff-477b-9199-69e58b73de7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:11.886873 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eb3e22-7bff-477b-9199-69e58b73de7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:11.887816 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eb3e22-7bff-477b-9199-69e58b73de7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:11.889848 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eb3e22-7bff-477b-9199-69e58b73de7f admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:11.902827 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a951ffd-53e5-42fa-936d-4528686ebefa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:11.903344 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 711/1424] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:20:11 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:20:11.905813 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a951ffd-53e5-42fa-936d-4528686ebefa service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:11.906241 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a951ffd-53e5-42fa-936d-4528686ebefa service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:11.906765 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a951ffd-53e5-42fa-936d-4528686ebefa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:11.908219 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eb3e22-7bff-477b-9199-69e58b73de7f admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:11.923486 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eb3e22-7bff-477b-9199-69e58b73de7f admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:11.929745 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7f7ac5-9eb0-42f9-b2ee-139e2c86a2f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:11.930496 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f7ac5-9eb0-42f9-b2ee-139e2c86a2f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:11.930726 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a951ffd-53e5-42fa-936d-4528686ebefa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:11.930832 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f7ac5-9eb0-42f9-b2ee-139e2c86a2f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:11.931200 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f7ac5-9eb0-42f9-b2ee-139e2c86a2f2 admin admin] PATH_INFO: `/v3/projects/238f3ac862444577beb163e70451deea/users/63a10287986a4c19a07bc674c642b0a2/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:11.933140 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a951ffd-53e5-42fa-936d-4528686ebefa service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:11.933886 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7f7ac5-9eb0-42f9-b2ee-139e2c86a2f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=238f3ac862444577beb163e70451deea, user_id=63a10287986a4c19a07bc674c642b0a2, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:11.937780 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7f7ac5-9eb0-42f9-b2ee-139e2c86a2f2 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:11.963481 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 712/1425] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:20:11 2022] PUT /identity/v3/projects/238f3ac862444577beb163e70451deea/users/63a10287986a4c19a07bc674c642b0a2/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:11.965942 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7f7ac5-9eb0-42f9-b2ee-139e2c86a2f2 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:11.966311 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 714/1426] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:20:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:20:11.980914 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e7485a-87e9-401b-9590-8aaf7add6b78 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:11.981195 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e7485a-87e9-401b-9590-8aaf7add6b78 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:11.981462 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e7485a-87e9-401b-9590-8aaf7add6b78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:12.051110 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-53e7485a-87e9-401b-9590-8aaf7add6b78 None None] MFA Rules not processed for user `63a10287986a4c19a07bc674c642b0a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:12.052630 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-53e7485a-87e9-401b-9590-8aaf7add6b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:12.090466 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7f7ac5-9eb0-42f9-b2ee-139e2c86a2f2 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:12.097748 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66769e57-f26f-4999-873b-a941ab2d513c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:12.099284 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66769e57-f26f-4999-873b-a941ab2d513c admin admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:12.099624 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66769e57-f26f-4999-873b-a941ab2d513c admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:12.099904 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66769e57-f26f-4999-873b-a941ab2d513c admin admin] PATH_INFO: `/v3/projects/9dd2567c3d2642e7ae57372262edd1f9/users/4796dae1f12f4163b0d0776e5fdd7903/roles/ef689f96442945e8a7bc8cbfee13d909` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:12.119303 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66769e57-f26f-4999-873b-a941ab2d513c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9dd2567c3d2642e7ae57372262edd1f9, user_id=4796dae1f12f4163b0d0776e5fdd7903, role_id=ef689f96442945e8a7bc8cbfee13d909)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:12.123509 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66769e57-f26f-4999-873b-a941ab2d513c admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:12.142326 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 715/1427] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:11 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:12.153945 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e7485a-87e9-401b-9590-8aaf7add6b78 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:12.156449 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-53e7485a-87e9-401b-9590-8aaf7add6b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:12.213724 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 713/1428] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:20:11 2022] PUT /identity/v3/projects/9dd2567c3d2642e7ae57372262edd1f9/users/4796dae1f12f4163b0d0776e5fdd7903/roles/ef689f96442945e8a7bc8cbfee13d909 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:12.226473 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a9d99c-c8cc-405e-a5f2-edf5b4576947 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:12.226856 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a9d99c-c8cc-405e-a5f2-edf5b4576947 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:12.227037 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a9d99c-c8cc-405e-a5f2-edf5b4576947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:12.288216 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-24a9d99c-c8cc-405e-a5f2-edf5b4576947 None None] MFA Rules not processed for user `4796dae1f12f4163b0d0776e5fdd7903`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:12.292252 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-24a9d99c-c8cc-405e-a5f2-edf5b4576947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:12.304520 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e7485a-87e9-401b-9590-8aaf7add6b78 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:12.415840 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 714/1429] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:12 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:12.417511 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a9d99c-c8cc-405e-a5f2-edf5b4576947 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:12.439518 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4ddd1a-9a14-4526-bfe2-21bfcad183c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:12.441773 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4ddd1a-9a14-4526-bfe2-21bfcad183c5 service cinder] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:12.444837 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4ddd1a-9a14-4526-bfe2-21bfcad183c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:12.445202 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4ddd1a-9a14-4526-bfe2-21bfcad183c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:12.506840 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:12.586792 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:12.642825 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4ddd1a-9a14-4526-bfe2-21bfcad183c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:12.645223 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4ddd1a-9a14-4526-bfe2-21bfcad183c5 service cinder] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:12.653207 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a9d99c-c8cc-405e-a5f2-edf5b4576947 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:12.665697 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 716/1430] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:20:12 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 00:20:12.666531 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cde3c0-60e3-4a2f-a1f8-4a1658e4b21c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:12.667190 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cde3c0-60e3-4a2f-a1f8-4a1658e4b21c admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:12.668081 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cde3c0-60e3-4a2f-a1f8-4a1658e4b21c admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:12.668374 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cde3c0-60e3-4a2f-a1f8-4a1658e4b21c admin admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:12.669816 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cde3c0-60e3-4a2f-a1f8-4a1658e4b21c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:12.671592 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cde3c0-60e3-4a2f-a1f8-4a1658e4b21c admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:12.824277 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 715/1431] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:20:12 2022] POST /identity/v3/projects => generated 363 bytes in 405 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:20:12.825379 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4ddd1a-9a14-4526-bfe2-21bfcad183c5 service cinder] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:12.841540 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4ddd1a-9a14-4526-bfe2-21bfcad183c5 service cinder] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:12.950539 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78388634-9e82-430d-bf7a-176873757295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:12.953300 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78388634-9e82-430d-bf7a-176873757295 admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:12.955300 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78388634-9e82-430d-bf7a-176873757295 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:12.956829 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78388634-9e82-430d-bf7a-176873757295 admin admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:12.957727 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78388634-9e82-430d-bf7a-176873757295 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:12.960502 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78388634-9e82-430d-bf7a-176873757295 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:13.026175 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cde3c0-60e3-4a2f-a1f8-4a1658e4b21c admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:13.028173 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-e8cde3c0-60e3-4a2f-a1f8-4a1658e4b21c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:13.116624 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 717/1432] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:20:12 2022] POST /identity/v3/users => generated 288 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:20:13.117710 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca128d1-8e04-43fb-a93a-8b82730cfd0d None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:13.117992 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca128d1-8e04-43fb-a93a-8b82730cfd0d None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:13.118262 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca128d1-8e04-43fb-a93a-8b82730cfd0d None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:13.123004 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 718/1433] 10.222.0.36 () {54 vars in 912 bytes} [Wed Oct 12 00:20:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:20:13.124497 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca128d1-8e04-43fb-a93a-8b82730cfd0d None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:13.173742 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca128d1-8e04-43fb-a93a-8b82730cfd0d None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:13.180273 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba53e9d-e0bf-43b5-9ed0-d717f8af37d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:13.180898 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba53e9d-e0bf-43b5-9ed0-d717f8af37d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:13.181205 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba53e9d-e0bf-43b5-9ed0-d717f8af37d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:13.181869 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba53e9d-e0bf-43b5-9ed0-d717f8af37d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:13.182574 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba53e9d-e0bf-43b5-9ed0-d717f8af37d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:13.184383 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba53e9d-e0bf-43b5-9ed0-d717f8af37d5 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:13.192657 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cde3c0-60e3-4a2f-a1f8-4a1658e4b21c admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:13.197354 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 719/1434] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:20:13 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:20:13.198639 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f266b73-e215-453e-9e09-5d2a9c3f246a None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:13.198924 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f266b73-e215-453e-9e09-5d2a9c3f246a None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:13.199287 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f266b73-e215-453e-9e09-5d2a9c3f246a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:13.216606 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af05e79a-a1d7-4154-b09f-c8fce1879716 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e528da752ed4c78bc747215909f6569', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:13.217922 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af05e79a-a1d7-4154-b09f-c8fce1879716 service placement] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:13.218295 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af05e79a-a1d7-4154-b09f-c8fce1879716 service placement] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:13.218659 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af05e79a-a1d7-4154-b09f-c8fce1879716 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:13.222661 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-af05e79a-a1d7-4154-b09f-c8fce1879716 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:13.259033 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af05e79a-a1d7-4154-b09f-c8fce1879716 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:13.267220 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af05e79a-a1d7-4154-b09f-c8fce1879716 service placement] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:13.273695 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-7f266b73-e215-453e-9e09-5d2a9c3f246a None None] MFA Rules not processed for user `51215dab618141b4991e232724094140`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:13.278261 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:13.278409 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-7f266b73-e215-453e-9e09-5d2a9c3f246a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:13.287867 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 716/1435] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:20:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 00:20:13.289507 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f035d7e-1fe5-4a4b-b53a-05b1f454c16f None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:13.290955 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f035d7e-1fe5-4a4b-b53a-05b1f454c16f None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:13.291404 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f035d7e-1fe5-4a4b-b53a-05b1f454c16f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:13.316988 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-0f035d7e-1fe5-4a4b-b53a-05b1f454c16f None None] MFA Rules not processed for user `cdd87151f3af4a3bb1c3815db5d0e8fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:13.319578 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-0f035d7e-1fe5-4a4b-b53a-05b1f454c16f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:13.387397 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:13.437794 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 720/1436] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Oct 12 00:20:13 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:13.455958 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 717/1437] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:13 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:13.456661 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f266b73-e215-453e-9e09-5d2a9c3f246a None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:13.458243 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f035d7e-1fe5-4a4b-b53a-05b1f454c16f None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:13.465404 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-0f035d7e-1fe5-4a4b-b53a-05b1f454c16f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:13.492656 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f266b73-e215-453e-9e09-5d2a9c3f246a None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:13.509603 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ebfbbb-e6e6-4519-8af5-4bb30ab82356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:13.510347 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ebfbbb-e6e6-4519-8af5-4bb30ab82356 admin admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:13.510621 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ebfbbb-e6e6-4519-8af5-4bb30ab82356 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:13.510925 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ebfbbb-e6e6-4519-8af5-4bb30ab82356 admin admin] PATH_INFO: `/v3/projects/58a4724798604fd789c15159e5e79ea0/users/2472c5efc0c1408da76d30191b5f7496/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:13.511389 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f035d7e-1fe5-4a4b-b53a-05b1f454c16f None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:13.519232 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1827ec72-784f-4376-83a2-63bfa9ec426e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:13.520015 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1827ec72-784f-4376-83a2-63bfa9ec426e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:13.520361 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1827ec72-784f-4376-83a2-63bfa9ec426e admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:13.520718 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1827ec72-784f-4376-83a2-63bfa9ec426e admin admin] PATH_INFO: `/v3/projects/f73ba417c10941a79c47a83f17bb5f19` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:13.521969 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1827ec72-784f-4376-83a2-63bfa9ec426e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f73ba417c10941a79c47a83f17bb5f19)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:13.524219 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1827ec72-784f-4376-83a2-63bfa9ec426e admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:13.546621 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ebfbbb-e6e6-4519-8af5-4bb30ab82356 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58a4724798604fd789c15159e5e79ea0, user_id=2472c5efc0c1408da76d30191b5f7496, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:13.551913 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ebfbbb-e6e6-4519-8af5-4bb30ab82356 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:13.632612 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 721/1438] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:20:13 2022] PUT /identity/v3/projects/58a4724798604fd789c15159e5e79ea0/users/2472c5efc0c1408da76d30191b5f7496/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:13.633481 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ebfbbb-e6e6-4519-8af5-4bb30ab82356 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:13.638354 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-a3ebfbbb-e6e6-4519-8af5-4bb30ab82356 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:13.679620 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-1827ec72-784f-4376-83a2-63bfa9ec426e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f73ba417c10941a79c47a83f17bb5f19', 'request_id': 'req-1827ec72-784f-4376-83a2-63bfa9ec426e'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:13.695128 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-1827ec72-784f-4376-83a2-63bfa9ec426e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f73ba417c10941a79c47a83f17bb5f19', 'request_id': 'req-1827ec72-784f-4376-83a2-63bfa9ec426e'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:13.820588 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-1827ec72-784f-4376-83a2-63bfa9ec426e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f73ba417c10941a79c47a83f17bb5f19', 'request_id': 'req-1827ec72-784f-4376-83a2-63bfa9ec426e'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:13.856821 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-1827ec72-784f-4376-83a2-63bfa9ec426e admin admin] The token cache is being invalidate because project f73ba417c10941a79c47a83f17bb5f19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81747) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:20:13.857881 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ebfbbb-e6e6-4519-8af5-4bb30ab82356 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:13.861175 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-1827ec72-784f-4376-83a2-63bfa9ec426e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:13.865160 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-1827ec72-784f-4376-83a2-63bfa9ec426e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:13.881306 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 718/1439] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:20:13 2022] DELETE /identity/v3/projects/f73ba417c10941a79c47a83f17bb5f19 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:13.881306 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cf3c84-782a-46c9-b61a-be9a3165eb2a None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:13.881306 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cf3c84-782a-46c9-b61a-be9a3165eb2a None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:13.881306 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cf3c84-782a-46c9-b61a-be9a3165eb2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:13.962575 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-17cf3c84-782a-46c9-b61a-be9a3165eb2a None None] MFA Rules not processed for user `bca53aca7b134df1949deb10858f14d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:13.977608 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-17cf3c84-782a-46c9-b61a-be9a3165eb2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:13.998030 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:14.051157 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bddd520-7d5a-4a4d-a721-05a964bca305 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:14.051816 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bddd520-7d5a-4a4d-a721-05a964bca305 service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:14.052092 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bddd520-7d5a-4a4d-a721-05a964bca305 service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:14.052366 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bddd520-7d5a-4a4d-a721-05a964bca305 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:14.057337 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-5bddd520-7d5a-4a4d-a721-05a964bca305 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:14.095130 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bddd520-7d5a-4a4d-a721-05a964bca305 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:14.097004 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bddd520-7d5a-4a4d-a721-05a964bca305 service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:14.130422 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 722/1440] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:20:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:20:14.133452 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bddd520-7d5a-4a4d-a721-05a964bca305 service neutron] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:14.136056 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-5bddd520-7d5a-4a4d-a721-05a964bca305 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:14.189114 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 719/1441] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:14.197760 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cf3c84-782a-46c9-b61a-be9a3165eb2a None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:14.206612 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-17cf3c84-782a-46c9-b61a-be9a3165eb2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:14.359075 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bddd520-7d5a-4a4d-a721-05a964bca305 service neutron] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:14.375890 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d363234-1a98-418c-9ef8-de6d0ad2819c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:14.379676 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d363234-1a98-418c-9ef8-de6d0ad2819c admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:14.380029 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d363234-1a98-418c-9ef8-de6d0ad2819c admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:14.381018 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d363234-1a98-418c-9ef8-de6d0ad2819c admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:14.381809 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d363234-1a98-418c-9ef8-de6d0ad2819c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:14.399076 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d363234-1a98-418c-9ef8-de6d0ad2819c admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:14.441648 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 723/1442] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:20:14 2022] GET /identity/v3/roles => generated 1420 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:20:14.443810 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cf3c84-782a-46c9-b61a-be9a3165eb2a None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:14.445971 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d363234-1a98-418c-9ef8-de6d0ad2819c admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:14.457484 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4f42fa-71b3-4cb9-b9ce-49ad4c662c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:14.471104 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4f42fa-71b3-4cb9-b9ce-49ad4c662c05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:14.471415 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4f42fa-71b3-4cb9-b9ce-49ad4c662c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:14.471705 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4f42fa-71b3-4cb9-b9ce-49ad4c662c05 admin admin] PATH_INFO: `/v3/users/7f946504b5574228bea5d7c88504e6f7` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:14.473137 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4f42fa-71b3-4cb9-b9ce-49ad4c662c05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f946504b5574228bea5d7c88504e6f7)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:14.483631 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4f42fa-71b3-4cb9-b9ce-49ad4c662c05 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:14.487914 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d363234-1a98-418c-9ef8-de6d0ad2819c admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:14.523291 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1343862-b903-465f-b254-2a7f67fcbee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:14.524413 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1343862-b903-465f-b254-2a7f67fcbee5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:14.524960 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1343862-b903-465f-b254-2a7f67fcbee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:14.525330 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1343862-b903-465f-b254-2a7f67fcbee5 admin admin] PATH_INFO: `/v3/projects/58a4724798604fd789c15159e5e79ea0/users/2472c5efc0c1408da76d30191b5f7496/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:14.529021 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1343862-b903-465f-b254-2a7f67fcbee5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58a4724798604fd789c15159e5e79ea0, user_id=2472c5efc0c1408da76d30191b5f7496, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:14.540549 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1343862-b903-465f-b254-2a7f67fcbee5 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:14.589978 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 724/1443] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:20:14 2022] PUT /identity/v3/projects/58a4724798604fd789c15159e5e79ea0/users/2472c5efc0c1408da76d30191b5f7496/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:14.606615 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78873307-c53c-4195-b1fa-5efd08eba2eb None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:14.606909 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78873307-c53c-4195-b1fa-5efd08eba2eb None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:14.618528 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78873307-c53c-4195-b1fa-5efd08eba2eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:14.717755 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-78873307-c53c-4195-b1fa-5efd08eba2eb None None] MFA Rules not processed for user `2472c5efc0c1408da76d30191b5f7496`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:14.721218 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-78873307-c53c-4195-b1fa-5efd08eba2eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:15.002630 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-af4f42fa-71b3-4cb9-b9ce-49ad4c662c05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f946504b5574228bea5d7c88504e6f7'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:15.005667 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-af4f42fa-71b3-4cb9-b9ce-49ad4c662c05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f946504b5574228bea5d7c88504e6f7'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:15.026612 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 725/1444] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:14 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 421 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:15.028979 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78873307-c53c-4195-b1fa-5efd08eba2eb None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:15.032878 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-78873307-c53c-4195-b1fa-5efd08eba2eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:15.033701 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-af4f42fa-71b3-4cb9-b9ce-49ad4c662c05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f946504b5574228bea5d7c88504e6f7'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:15.106836 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-af4f42fa-71b3-4cb9-b9ce-49ad4c662c05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f946504b5574228bea5d7c88504e6f7'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:15.110018 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-af4f42fa-71b3-4cb9-b9ce-49ad4c662c05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f946504b5574228bea5d7c88504e6f7'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:15.161139 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 720/1445] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:20:14 2022] DELETE /identity/v3/users/7f946504b5574228bea5d7c88504e6f7 => generated 0 bytes in 971 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:15.225074 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78873307-c53c-4195-b1fa-5efd08eba2eb None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:15.381167 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe5dfe3-16a1-4ade-8775-990464dc14a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:15.381167 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe5dfe3-16a1-4ade-8775-990464dc14a4 service cinder] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:15.381167 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe5dfe3-16a1-4ade-8775-990464dc14a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:15.381167 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe5dfe3-16a1-4ade-8775-990464dc14a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:15.385063 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-5fe5dfe3-16a1-4ade-8775-990464dc14a4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:15.577069 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe5dfe3-16a1-4ade-8775-990464dc14a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:15.580976 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe5dfe3-16a1-4ade-8775-990464dc14a4 service cinder] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:15.618763 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 726/1446] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:20:15 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 591 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:20:15.962554 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4f42fa-71b3-4cb9-b9ce-49ad4c662c05 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:15.964286 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-af4f42fa-71b3-4cb9-b9ce-49ad4c662c05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:15.993858 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4f42fa-71b3-4cb9-b9ce-49ad4c662c05 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:16.008952 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d7403b-86d1-4551-bd16-844ce490f5ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:16.009584 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d7403b-86d1-4551-bd16-844ce490f5ac service cinder] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:16.009848 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d7403b-86d1-4551-bd16-844ce490f5ac service cinder] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:16.010113 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d7403b-86d1-4551-bd16-844ce490f5ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:16.012088 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-b8d7403b-86d1-4551-bd16-844ce490f5ac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:16.175163 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:16.200925 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d7403b-86d1-4551-bd16-844ce490f5ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:16.206242 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d7403b-86d1-4551-bd16-844ce490f5ac service cinder] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:16.230106 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 721/1447] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:20:15 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:20:16.318154 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe5dfe3-16a1-4ade-8775-990464dc14a4 service cinder] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:16.324904 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-5fe5dfe3-16a1-4ade-8775-990464dc14a4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:16.413910 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:16.478612 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe5dfe3-16a1-4ade-8775-990464dc14a4 service cinder] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:16.487473 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e482ec85-7ff9-46d7-8d49-b59cba2ee1e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:16.488972 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e482ec85-7ff9-46d7-8d49-b59cba2ee1e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:16.489393 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e482ec85-7ff9-46d7-8d49-b59cba2ee1e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:16.489802 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e482ec85-7ff9-46d7-8d49-b59cba2ee1e2 admin admin] PATH_INFO: `/v3/projects/60c8fb2d6b1a4f6696e901317fd99ae7` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:16.491611 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e482ec85-7ff9-46d7-8d49-b59cba2ee1e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60c8fb2d6b1a4f6696e901317fd99ae7)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:16.493655 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e482ec85-7ff9-46d7-8d49-b59cba2ee1e2 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:16.770788 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-e482ec85-7ff9-46d7-8d49-b59cba2ee1e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60c8fb2d6b1a4f6696e901317fd99ae7', 'request_id': 'req-e482ec85-7ff9-46d7-8d49-b59cba2ee1e2'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:16.788149 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-e482ec85-7ff9-46d7-8d49-b59cba2ee1e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60c8fb2d6b1a4f6696e901317fd99ae7', 'request_id': 'req-e482ec85-7ff9-46d7-8d49-b59cba2ee1e2'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:16.815611 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784021b4-00c5-4943-a921-64b0f5628db4 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:16.815943 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784021b4-00c5-4943-a921-64b0f5628db4 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:16.816154 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784021b4-00c5-4943-a921-64b0f5628db4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:16.848388 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-784021b4-00c5-4943-a921-64b0f5628db4 None None] MFA Rules not processed for user `de766cb582ba4c18b1fda9bf4ad43f65`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:16.849612 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-784021b4-00c5-4943-a921-64b0f5628db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:16.941132 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 722/1448] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 00:20:16.952247 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-e482ec85-7ff9-46d7-8d49-b59cba2ee1e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60c8fb2d6b1a4f6696e901317fd99ae7', 'request_id': 'req-e482ec85-7ff9-46d7-8d49-b59cba2ee1e2'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:16.974627 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-e482ec85-7ff9-46d7-8d49-b59cba2ee1e2 admin admin] The token cache is being invalidate because project 60c8fb2d6b1a4f6696e901317fd99ae7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81746) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:20:16.974929 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-e482ec85-7ff9-46d7-8d49-b59cba2ee1e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:16.976548 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-e482ec85-7ff9-46d7-8d49-b59cba2ee1e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:17.009405 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 727/1449] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:20:16 2022] DELETE /identity/v3/projects/60c8fb2d6b1a4f6696e901317fd99ae7 => generated 0 bytes in 692 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:17.016525 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66836ed7-962b-4bef-9066-9926fdb4b844 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:17.016981 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66836ed7-962b-4bef-9066-9926fdb4b844 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:17.017094 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66836ed7-962b-4bef-9066-9926fdb4b844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:17.048075 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-66836ed7-962b-4bef-9066-9926fdb4b844 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:17.051587 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-66836ed7-962b-4bef-9066-9926fdb4b844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:17.194802 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 723/1450] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:17.195234 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e33afa5-f318-4645-8bd6-858cfba9d1b6 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:17.195560 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e33afa5-f318-4645-8bd6-858cfba9d1b6 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:17.195839 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e33afa5-f318-4645-8bd6-858cfba9d1b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:17.291127 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-8e33afa5-f318-4645-8bd6-858cfba9d1b6 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:17.292411 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-8e33afa5-f318-4645-8bd6-858cfba9d1b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:17.336972 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 728/1451] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:17.338396 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66836ed7-962b-4bef-9066-9926fdb4b844 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:17.365080 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66836ed7-962b-4bef-9066-9926fdb4b844 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:17.386784 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8ce7ef-d872-4732-8294-74b50da095b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:17.390210 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ce7ef-d872-4732-8294-74b50da095b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:17.390564 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ce7ef-d872-4732-8294-74b50da095b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:17.391431 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ce7ef-d872-4732-8294-74b50da095b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:17.393083 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8ce7ef-d872-4732-8294-74b50da095b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:20:17.395265 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8ce7ef-d872-4732-8294-74b50da095b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:17.405606 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8ce7ef-d872-4732-8294-74b50da095b6 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:17.453909 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 724/1452] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Oct 12 00:20:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:20:17.459270 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e33afa5-f318-4645-8bd6-858cfba9d1b6 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:17.472622 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e33afa5-f318-4645-8bd6-858cfba9d1b6 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:17.518435 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ee8d00-1948-4b60-8108-2f66555aa06a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:17.519732 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ee8d00-1948-4b60-8108-2f66555aa06a admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:17.520023 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ee8d00-1948-4b60-8108-2f66555aa06a admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:17.520301 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ee8d00-1948-4b60-8108-2f66555aa06a admin admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:17.524698 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ee8d00-1948-4b60-8108-2f66555aa06a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:17.532178 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ee8d00-1948-4b60-8108-2f66555aa06a admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:17.748691 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 729/1453] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:20:17 2022] POST /identity/v3/projects => generated 363 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:20:17.752987 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8ce7ef-d872-4732-8294-74b50da095b6 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:17.770401 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8ce7ef-d872-4732-8294-74b50da095b6 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:17.894802 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fc761a-ed2f-4d4c-ad6d-d3d42de51d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:17.896016 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fc761a-ed2f-4d4c-ad6d-d3d42de51d99 admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:17.896547 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fc761a-ed2f-4d4c-ad6d-d3d42de51d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:17.897030 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fc761a-ed2f-4d4c-ad6d-d3d42de51d99 admin admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:17.897981 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fc761a-ed2f-4d4c-ad6d-d3d42de51d99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:17.900135 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fc761a-ed2f-4d4c-ad6d-d3d42de51d99 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:18.024053 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ee8d00-1948-4b60-8108-2f66555aa06a admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:18.026252 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-36ee8d00-1948-4b60-8108-2f66555aa06a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:18.048630 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 725/1454] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:20:17 2022] POST /identity/v3/users => generated 282 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:20:18.055677 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fc761a-ed2f-4d4c-ad6d-d3d42de51d99 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:18.074532 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ee8d00-1948-4b60-8108-2f66555aa06a admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:18.075854 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fc761a-ed2f-4d4c-ad6d-d3d42de51d99 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:18.082568 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6464a8ed-49c1-4591-a115-4144459e963c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:18.083744 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6464a8ed-49c1-4591-a115-4144459e963c admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:18.084191 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6464a8ed-49c1-4591-a115-4144459e963c admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:18.084813 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6464a8ed-49c1-4591-a115-4144459e963c admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:18.086467 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6464a8ed-49c1-4591-a115-4144459e963c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:18.088350 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6464a8ed-49c1-4591-a115-4144459e963c admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:18.093018 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5077d290-4907-4a61-8e66-6c7aeeac2f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:18.093732 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5077d290-4907-4a61-8e66-6c7aeeac2f88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:18.094003 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5077d290-4907-4a61-8e66-6c7aeeac2f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:18.095620 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5077d290-4907-4a61-8e66-6c7aeeac2f88 admin admin] PATH_INFO: `/v3/users/1fc758a783414aa588ec3604ebed8970` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:18.099590 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5077d290-4907-4a61-8e66-6c7aeeac2f88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fc758a783414aa588ec3604ebed8970)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:18.105967 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5077d290-4907-4a61-8e66-6c7aeeac2f88 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:18.135327 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 726/1455] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:20:18 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 00:20:18.139683 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6464a8ed-49c1-4591-a115-4144459e963c admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:18.166482 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6464a8ed-49c1-4591-a115-4144459e963c admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:18.186625 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c43bdd-418b-4e21-ae08-2e66741584ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:18.187653 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c43bdd-418b-4e21-ae08-2e66741584ee admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:18.190292 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c43bdd-418b-4e21-ae08-2e66741584ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:18.190739 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c43bdd-418b-4e21-ae08-2e66741584ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:18.191625 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c43bdd-418b-4e21-ae08-2e66741584ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:18.195697 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c43bdd-418b-4e21-ae08-2e66741584ee admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:18.222791 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 727/1456] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:20:18 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:20:18.226949 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c43bdd-418b-4e21-ae08-2e66741584ee admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:18.258665 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c43bdd-418b-4e21-ae08-2e66741584ee admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:18.274027 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9892537d-10fc-4b36-a0c9-36a11855767e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:18.274909 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9892537d-10fc-4b36-a0c9-36a11855767e admin admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:18.275396 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9892537d-10fc-4b36-a0c9-36a11855767e admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:18.279270 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9892537d-10fc-4b36-a0c9-36a11855767e admin admin] PATH_INFO: `/v3/projects/1761e198d6d44288807d407cdcec48c5/users/716cc08f22ec4f36931f3fda7a137a58/roles/ef689f96442945e8a7bc8cbfee13d909` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:18.309589 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9892537d-10fc-4b36-a0c9-36a11855767e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1761e198d6d44288807d407cdcec48c5, user_id=716cc08f22ec4f36931f3fda7a137a58, role_id=ef689f96442945e8a7bc8cbfee13d909)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:18.312836 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9892537d-10fc-4b36-a0c9-36a11855767e admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:18.412227 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:18.428125 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 728/1457] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:20:18 2022] PUT /identity/v3/projects/1761e198d6d44288807d407cdcec48c5/users/716cc08f22ec4f36931f3fda7a137a58/roles/ef689f96442945e8a7bc8cbfee13d909 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:18.438745 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9581313d-ee72-4a46-9b51-ad862d9ffa39 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:18.439120 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9581313d-ee72-4a46-9b51-ad862d9ffa39 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:18.439492 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9581313d-ee72-4a46-9b51-ad862d9ffa39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:18.475488 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-5077d290-4907-4a61-8e66-6c7aeeac2f88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fc758a783414aa588ec3604ebed8970'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:18.476621 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-5077d290-4907-4a61-8e66-6c7aeeac2f88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fc758a783414aa588ec3604ebed8970'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:18.512545 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-5077d290-4907-4a61-8e66-6c7aeeac2f88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fc758a783414aa588ec3604ebed8970'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:18.523504 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:18.548931 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-9581313d-ee72-4a46-9b51-ad862d9ffa39 None None] MFA Rules not processed for user `716cc08f22ec4f36931f3fda7a137a58`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:18.556533 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-9581313d-ee72-4a46-9b51-ad862d9ffa39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:18.677658 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-5077d290-4907-4a61-8e66-6c7aeeac2f88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fc758a783414aa588ec3604ebed8970'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:18.678581 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-5077d290-4907-4a61-8e66-6c7aeeac2f88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fc758a783414aa588ec3604ebed8970'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:18.710411 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 730/1458] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:20:18 2022] DELETE /identity/v3/users/1fc758a783414aa588ec3604ebed8970 => generated 0 bytes in 688 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:18.714938 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc3f47-fe01-4631-9200-9ea9ad6c6982 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:18.717410 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc3f47-fe01-4631-9200-9ea9ad6c6982 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:18.717708 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc3f47-fe01-4631-9200-9ea9ad6c6982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:18.739498 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 729/1459] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:18 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:18.751397 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9581313d-ee72-4a46-9b51-ad862d9ffa39 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:18.751397 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-9581313d-ee72-4a46-9b51-ad862d9ffa39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:18.784872 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-d4fc3f47-fe01-4631-9200-9ea9ad6c6982 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:18.787105 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-d4fc3f47-fe01-4631-9200-9ea9ad6c6982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:18.968319 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9581313d-ee72-4a46-9b51-ad862d9ffa39 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:18.983897 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 731/1460] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:18.990123 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aba480-c0a1-4c6e-9b93-6536efc6672b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:18.990942 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aba480-c0a1-4c6e-9b93-6536efc6672b admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:18.991352 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aba480-c0a1-4c6e-9b93-6536efc6672b admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:18.991773 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aba480-c0a1-4c6e-9b93-6536efc6672b admin admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:18.992575 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aba480-c0a1-4c6e-9b93-6536efc6672b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:18.994112 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fc3f47-fe01-4631-9200-9ea9ad6c6982 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:18.999124 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aba480-c0a1-4c6e-9b93-6536efc6672b admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:19.001757 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-d4fc3f47-fe01-4631-9200-9ea9ad6c6982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:19.145348 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fc3f47-fe01-4631-9200-9ea9ad6c6982 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:19.149324 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e861e67e-4e8c-4bc0-90af-2f215d882771 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:19.149945 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e861e67e-4e8c-4bc0-90af-2f215d882771 service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:19.151020 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e861e67e-4e8c-4bc0-90af-2f215d882771 service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:19.152037 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e861e67e-4e8c-4bc0-90af-2f215d882771 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:19.168420 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e861e67e-4e8c-4bc0-90af-2f215d882771 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:19.173420 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e861e67e-4e8c-4bc0-90af-2f215d882771 service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:19.183353 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 730/1461] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:20:18 2022] POST /identity/v3/projects => generated 365 bytes in 440 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:20:19.187603 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aba480-c0a1-4c6e-9b93-6536efc6672b admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:19.198533 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 732/1462] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:20:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:20:19.300679 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aba480-c0a1-4c6e-9b93-6536efc6672b admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:19.306648 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc04fbea-c9e3-4afe-a1b3-8d4837f4b114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:19.307806 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc04fbea-c9e3-4afe-a1b3-8d4837f4b114 admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:19.308402 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc04fbea-c9e3-4afe-a1b3-8d4837f4b114 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:19.309102 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc04fbea-c9e3-4afe-a1b3-8d4837f4b114 admin admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:19.310364 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc04fbea-c9e3-4afe-a1b3-8d4837f4b114 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:19.312618 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc04fbea-c9e3-4afe-a1b3-8d4837f4b114 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:19.421254 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 731/1463] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:20:19 2022] POST /identity/v3/users => generated 289 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 00:20:19.426054 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e861e67e-4e8c-4bc0-90af-2f215d882771 service neutron] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:19.437685 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e861e67e-4e8c-4bc0-90af-2f215d882771 service neutron] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:19.443821 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0705e58-bb33-40f8-ae31-97a846c7ba4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:19.444817 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0705e58-bb33-40f8-ae31-97a846c7ba4e admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:19.450383 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0705e58-bb33-40f8-ae31-97a846c7ba4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:19.450383 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0705e58-bb33-40f8-ae31-97a846c7ba4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:19.450383 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0705e58-bb33-40f8-ae31-97a846c7ba4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:19.450383 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0705e58-bb33-40f8-ae31-97a846c7ba4e admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:19.472853 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 733/1464] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:20:19 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:20:19.481333 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc04fbea-c9e3-4afe-a1b3-8d4837f4b114 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:19.524249 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc04fbea-c9e3-4afe-a1b3-8d4837f4b114 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:19.548557 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56196cf-e383-4ccb-b30d-d5aacbb13618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:19.550748 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56196cf-e383-4ccb-b30d-d5aacbb13618 admin admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:19.551707 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56196cf-e383-4ccb-b30d-d5aacbb13618 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:19.554316 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56196cf-e383-4ccb-b30d-d5aacbb13618 admin admin] PATH_INFO: `/v3/projects/83468573643c456187416060e3b65170/users/0da7140e0e70415b9e6b453ec71546da/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:19.620534 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56196cf-e383-4ccb-b30d-d5aacbb13618 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83468573643c456187416060e3b65170, user_id=0da7140e0e70415b9e6b453ec71546da, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:19.654079 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56196cf-e383-4ccb-b30d-d5aacbb13618 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:19.776702 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 732/1465] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:20:19 2022] PUT /identity/v3/projects/83468573643c456187416060e3b65170/users/0da7140e0e70415b9e6b453ec71546da/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:19.781252 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0705e58-bb33-40f8-ae31-97a846c7ba4e admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:19.811853 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0705e58-bb33-40f8-ae31-97a846c7ba4e admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:19.953347 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4ef0a4-07c3-4a52-9611-618b07f4fdcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:19.955423 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4ef0a4-07c3-4a52-9611-618b07f4fdcf admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:19.956518 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4ef0a4-07c3-4a52-9611-618b07f4fdcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:19.956938 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4ef0a4-07c3-4a52-9611-618b07f4fdcf admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:19.957743 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4ef0a4-07c3-4a52-9611-618b07f4fdcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:19.961479 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4ef0a4-07c3-4a52-9611-618b07f4fdcf admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:19.984465 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56196cf-e383-4ccb-b30d-d5aacbb13618 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:19.986282 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-d56196cf-e383-4ccb-b30d-d5aacbb13618 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:19.986431 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 734/1466] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:20:19 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:20:19.990721 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4ef0a4-07c3-4a52-9611-618b07f4fdcf admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:20.005015 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4ef0a4-07c3-4a52-9611-618b07f4fdcf admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:20.014460 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbae0547-f4c5-4034-b0ca-31f0feb879a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:20.015608 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbae0547-f4c5-4034-b0ca-31f0feb879a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:20.016047 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbae0547-f4c5-4034-b0ca-31f0feb879a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:20.016871 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbae0547-f4c5-4034-b0ca-31f0feb879a7 admin admin] PATH_INFO: `/v3/projects/83468573643c456187416060e3b65170/users/0da7140e0e70415b9e6b453ec71546da/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:20.020068 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbae0547-f4c5-4034-b0ca-31f0feb879a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83468573643c456187416060e3b65170, user_id=0da7140e0e70415b9e6b453ec71546da, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:20.024394 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56196cf-e383-4ccb-b30d-d5aacbb13618 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:20.024614 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbae0547-f4c5-4034-b0ca-31f0feb879a7 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:20.038216 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 735/1467] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:20:19 2022] PUT /identity/v3/projects/83468573643c456187416060e3b65170/users/0da7140e0e70415b9e6b453ec71546da/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 00:20:20.050392 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfa44a2-bb3e-4308-a524-ecf1c2cc190c None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:20.050866 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfa44a2-bb3e-4308-a524-ecf1c2cc190c None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:20.051921 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfa44a2-bb3e-4308-a524-ecf1c2cc190c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:20.171939 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18914810-2f6c-465b-af83-a111d239ee48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:20.172858 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18914810-2f6c-465b-af83-a111d239ee48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:20.176767 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18914810-2f6c-465b-af83-a111d239ee48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:20.176767 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18914810-2f6c-465b-af83-a111d239ee48 admin admin] PATH_INFO: `/v3/projects/1f81357071c84cbeac45598ec351bed8` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:20.176767 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18914810-2f6c-465b-af83-a111d239ee48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f81357071c84cbeac45598ec351bed8)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:20.177294 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18914810-2f6c-465b-af83-a111d239ee48 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:20.180366 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-8bfa44a2-bb3e-4308-a524-ecf1c2cc190c None None] MFA Rules not processed for user `0da7140e0e70415b9e6b453ec71546da`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:20.183533 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-8bfa44a2-bb3e-4308-a524-ecf1c2cc190c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:20.375483 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 736/1468] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:20 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:20.383294 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-18914810-2f6c-465b-af83-a111d239ee48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f81357071c84cbeac45598ec351bed8', 'request_id': 'req-18914810-2f6c-465b-af83-a111d239ee48'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:20.387501 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfa44a2-bb3e-4308-a524-ecf1c2cc190c None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:20.390056 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-8bfa44a2-bb3e-4308-a524-ecf1c2cc190c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:20.393735 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-18914810-2f6c-465b-af83-a111d239ee48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f81357071c84cbeac45598ec351bed8', 'request_id': 'req-18914810-2f6c-465b-af83-a111d239ee48'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:20.462848 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-18914810-2f6c-465b-af83-a111d239ee48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f81357071c84cbeac45598ec351bed8', 'request_id': 'req-18914810-2f6c-465b-af83-a111d239ee48'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:20.470567 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfa44a2-bb3e-4308-a524-ecf1c2cc190c None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:20.475937 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-18914810-2f6c-465b-af83-a111d239ee48 admin admin] The token cache is being invalidate because project 1f81357071c84cbeac45598ec351bed8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81747) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:20:20.475937 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-18914810-2f6c-465b-af83-a111d239ee48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:20.476868 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-18914810-2f6c-465b-af83-a111d239ee48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:20.478320 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 733/1469] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:20:19 2022] DELETE /identity/v3/projects/1f81357071c84cbeac45598ec351bed8 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:20.482854 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18914810-2f6c-465b-af83-a111d239ee48 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:20.484546 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-18914810-2f6c-465b-af83-a111d239ee48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:20.528705 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690fa79a-0b19-4748-9772-c7045b00f7aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:20.530220 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690fa79a-0b19-4748-9772-c7045b00f7aa service cinder] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:20.530220 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690fa79a-0b19-4748-9772-c7045b00f7aa service cinder] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:20.544319 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690fa79a-0b19-4748-9772-c7045b00f7aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:20.550259 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-690fa79a-0b19-4748-9772-c7045b00f7aa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:20.591113 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18914810-2f6c-465b-af83-a111d239ee48 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:20.600214 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1770080-9f25-4b16-9f71-8ce618955f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:20.601114 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1770080-9f25-4b16-9f71-8ce618955f7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:20.601561 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1770080-9f25-4b16-9f71-8ce618955f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:20.602000 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1770080-9f25-4b16-9f71-8ce618955f7f admin admin] PATH_INFO: `/v3/users/63a10287986a4c19a07bc674c642b0a2` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:20.603909 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1770080-9f25-4b16-9f71-8ce618955f7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=63a10287986a4c19a07bc674c642b0a2)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:20.609864 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1770080-9f25-4b16-9f71-8ce618955f7f admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:20.652561 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690fa79a-0b19-4748-9772-c7045b00f7aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:20.656737 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690fa79a-0b19-4748-9772-c7045b00f7aa service cinder] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:20.671680 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 737/1470] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:20:20 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:20:20.766316 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:20.825238 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-f1770080-9f25-4b16-9f71-8ce618955f7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63a10287986a4c19a07bc674c642b0a2'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:20.826090 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-f1770080-9f25-4b16-9f71-8ce618955f7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63a10287986a4c19a07bc674c642b0a2'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:20.841533 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-f1770080-9f25-4b16-9f71-8ce618955f7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63a10287986a4c19a07bc674c642b0a2'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:20.914688 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-f1770080-9f25-4b16-9f71-8ce618955f7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63a10287986a4c19a07bc674c642b0a2'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:20.917278 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-f1770080-9f25-4b16-9f71-8ce618955f7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63a10287986a4c19a07bc674c642b0a2'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:20.949597 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 734/1471] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:20:20 2022] DELETE /identity/v3/users/63a10287986a4c19a07bc674c642b0a2 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:20.951271 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690fa79a-0b19-4748-9772-c7045b00f7aa service cinder] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:20.956639 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-690fa79a-0b19-4748-9772-c7045b00f7aa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:20.972001 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690fa79a-0b19-4748-9772-c7045b00f7aa service cinder] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:21.007562 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af5475-625e-4dfc-af23-07d2034c8b76 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:21.008062 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af5475-625e-4dfc-af23-07d2034c8b76 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:21.008430 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af5475-625e-4dfc-af23-07d2034c8b76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:21.089618 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a6dcf4-1d07-4414-b13b-5270bfea42f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:21.090443 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a6dcf4-1d07-4414-b13b-5270bfea42f9 service cinder] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:21.090896 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a6dcf4-1d07-4414-b13b-5270bfea42f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:21.091334 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a6dcf4-1d07-4414-b13b-5270bfea42f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:21.095973 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-72a6dcf4-1d07-4414-b13b-5270bfea42f9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:21.116900 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-d3af5475-625e-4dfc-af23-07d2034c8b76 None None] MFA Rules not processed for user `d030d18b595144f19d6ad01d4cd5d52b`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:21.119245 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-d3af5475-625e-4dfc-af23-07d2034c8b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:21.217798 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a6dcf4-1d07-4414-b13b-5270bfea42f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:21.219274 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a6dcf4-1d07-4414-b13b-5270bfea42f9 service cinder] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:21.246939 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 738/1472] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:20:20 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:20:21.355518 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 735/1473] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Oct 12 00:20:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:21.459335 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:21.688774 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:21.858959 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a6dcf4-1d07-4414-b13b-5270bfea42f9 service cinder] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:21.860610 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-72a6dcf4-1d07-4414-b13b-5270bfea42f9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:22.177665 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a6dcf4-1d07-4414-b13b-5270bfea42f9 service cinder] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:22.218430 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303f4337-c933-4743-a496-d099eae8462c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc1bcffc78844dcbb1a9b7c80285249b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:22.219257 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303f4337-c933-4743-a496-d099eae8462c service swift] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:22.220490 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303f4337-c933-4743-a496-d099eae8462c service swift] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:22.221771 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303f4337-c933-4743-a496-d099eae8462c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:22.251484 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303f4337-c933-4743-a496-d099eae8462c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:22.254493 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303f4337-c933-4743-a496-d099eae8462c service swift] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:22.280216 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 739/1474] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Oct 12 00:20:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 12 00:20:22.338280 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c30585-87a7-46bf-ada0-b20d93e111a8 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:22.338741 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c30585-87a7-46bf-ada0-b20d93e111a8 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:22.339178 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c30585-87a7-46bf-ada0-b20d93e111a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:22.405706 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-b8c30585-87a7-46bf-ada0-b20d93e111a8 None None] MFA Rules not processed for user `51215dab618141b4991e232724094140`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:22.412458 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-b8c30585-87a7-46bf-ada0-b20d93e111a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:22.650719 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 736/1475] 10.222.0.40 () {58 vars in 1015 bytes} [Wed Oct 12 00:20:22 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:22.711172 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303f4337-c933-4743-a496-d099eae8462c service swift] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:22.713250 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-303f4337-c933-4743-a496-d099eae8462c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:22.733295 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303f4337-c933-4743-a496-d099eae8462c service swift] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:22.767060 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741831ca-b4b5-4676-a656-637f6565820a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:22.778471 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741831ca-b4b5-4676-a656-637f6565820a service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:22.778471 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741831ca-b4b5-4676-a656-637f6565820a service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:22.778471 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741831ca-b4b5-4676-a656-637f6565820a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:22.798398 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741831ca-b4b5-4676-a656-637f6565820a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:22.800276 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741831ca-b4b5-4676-a656-637f6565820a service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:22.831069 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 740/1476] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:20:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:20:22.912797 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:23.243252 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:24.034567 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c30585-87a7-46bf-ada0-b20d93e111a8 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:24.036986 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-b8c30585-87a7-46bf-ada0-b20d93e111a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:24.228506 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c30585-87a7-46bf-ada0-b20d93e111a8 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:24.249535 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13017e3a-ce2e-4e6c-9a72-42feca72ad6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:24.250348 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13017e3a-ce2e-4e6c-9a72-42feca72ad6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:24.250625 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13017e3a-ce2e-4e6c-9a72-42feca72ad6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:24.250971 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13017e3a-ce2e-4e6c-9a72-42feca72ad6b admin admin] PATH_INFO: `/v3/projects/238f3ac862444577beb163e70451deea` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:24.252293 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13017e3a-ce2e-4e6c-9a72-42feca72ad6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=238f3ac862444577beb163e70451deea)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:24.256775 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13017e3a-ce2e-4e6c-9a72-42feca72ad6b admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:24.489731 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-13017e3a-ce2e-4e6c-9a72-42feca72ad6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '238f3ac862444577beb163e70451deea', 'request_id': 'req-13017e3a-ce2e-4e6c-9a72-42feca72ad6b'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:24.533875 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-13017e3a-ce2e-4e6c-9a72-42feca72ad6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '238f3ac862444577beb163e70451deea', 'request_id': 'req-13017e3a-ce2e-4e6c-9a72-42feca72ad6b'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:24.637162 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-13017e3a-ce2e-4e6c-9a72-42feca72ad6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '238f3ac862444577beb163e70451deea', 'request_id': 'req-13017e3a-ce2e-4e6c-9a72-42feca72ad6b'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:24.655041 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-13017e3a-ce2e-4e6c-9a72-42feca72ad6b admin admin] The token cache is being invalidate because project 238f3ac862444577beb163e70451deea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81747) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:20:24.655502 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-13017e3a-ce2e-4e6c-9a72-42feca72ad6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:24.667071 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-13017e3a-ce2e-4e6c-9a72-42feca72ad6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:24.675122 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 737/1477] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:20:24 2022] DELETE /identity/v3/projects/238f3ac862444577beb163e70451deea => generated 0 bytes in 641 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:24.680093 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018b49e7-e628-44e6-bd2d-45d74033ada9 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:24.680588 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018b49e7-e628-44e6-bd2d-45d74033ada9 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:24.680668 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018b49e7-e628-44e6-bd2d-45d74033ada9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:24.747624 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-018b49e7-e628-44e6-bd2d-45d74033ada9 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:24.749852 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-018b49e7-e628-44e6-bd2d-45d74033ada9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:24.873110 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:24.892272 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 741/1478] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:24.894072 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1feeea-3ab2-4a9d-86c5-9e15de65016e None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:24.894363 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1feeea-3ab2-4a9d-86c5-9e15de65016e None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:24.894748 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1feeea-3ab2-4a9d-86c5-9e15de65016e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:24.927579 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-ac1feeea-3ab2-4a9d-86c5-9e15de65016e None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:24.929399 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-ac1feeea-3ab2-4a9d-86c5-9e15de65016e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:24.938834 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018b49e7-e628-44e6-bd2d-45d74033ada9 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:24.940847 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-018b49e7-e628-44e6-bd2d-45d74033ada9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:24.951771 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 738/1479] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:24.956420 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1feeea-3ab2-4a9d-86c5-9e15de65016e None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:24.962058 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018b49e7-e628-44e6-bd2d-45d74033ada9 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:24.983625 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11585141-5840-4bfc-8121-29db53e3c10f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:24.984549 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1feeea-3ab2-4a9d-86c5-9e15de65016e None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:24.984716 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11585141-5840-4bfc-8121-29db53e3c10f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:24.985107 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11585141-5840-4bfc-8121-29db53e3c10f admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:24.985594 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11585141-5840-4bfc-8121-29db53e3c10f admin admin] PATH_INFO: `/v3/users/c24aa3f7f951483d991e59c93f01ce49` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:24.987062 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11585141-5840-4bfc-8121-29db53e3c10f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c24aa3f7f951483d991e59c93f01ce49)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:24.999284 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11585141-5840-4bfc-8121-29db53e3c10f admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:24.999491 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8686fac-d76f-488a-a141-11cb406685cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:24.999491 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8686fac-d76f-488a-a141-11cb406685cc admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:24.999491 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8686fac-d76f-488a-a141-11cb406685cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:24.999711 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8686fac-d76f-488a-a141-11cb406685cc admin admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:25.000467 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8686fac-d76f-488a-a141-11cb406685cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:20:25.000809 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8686fac-d76f-488a-a141-11cb406685cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:25.004622 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8686fac-d76f-488a-a141-11cb406685cc admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:25.032867 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 739/1480] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Oct 12 00:20:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:20:25.033721 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8686fac-d76f-488a-a141-11cb406685cc admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:25.052371 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:25.080922 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8686fac-d76f-488a-a141-11cb406685cc admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:25.089420 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedbf391-5a5a-4e22-9810-c580f0ee620d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:25.094010 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedbf391-5a5a-4e22-9810-c580f0ee620d admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:25.094558 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedbf391-5a5a-4e22-9810-c580f0ee620d admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:25.094993 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedbf391-5a5a-4e22-9810-c580f0ee620d admin admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:25.095990 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedbf391-5a5a-4e22-9810-c580f0ee620d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:25.098951 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedbf391-5a5a-4e22-9810-c580f0ee620d admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:25.255616 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 740/1481] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:20:25 2022] POST /identity/v3/projects => generated 367 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:20:25.260279 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedbf391-5a5a-4e22-9810-c580f0ee620d admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:25.297087 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-11585141-5840-4bfc-8121-29db53e3c10f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c24aa3f7f951483d991e59c93f01ce49'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:25.298138 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-11585141-5840-4bfc-8121-29db53e3c10f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c24aa3f7f951483d991e59c93f01ce49'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:25.314364 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-11585141-5840-4bfc-8121-29db53e3c10f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c24aa3f7f951483d991e59c93f01ce49'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:25.386648 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-11585141-5840-4bfc-8121-29db53e3c10f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c24aa3f7f951483d991e59c93f01ce49'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:25.387279 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-11585141-5840-4bfc-8121-29db53e3c10f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c24aa3f7f951483d991e59c93f01ce49'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:25.392023 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedbf391-5a5a-4e22-9810-c580f0ee620d admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:25.412285 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9dda84-6969-4385-ad2a-b041a4bd3119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:25.412939 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9dda84-6969-4385-ad2a-b041a4bd3119 admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:25.413506 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9dda84-6969-4385-ad2a-b041a4bd3119 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:25.413506 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9dda84-6969-4385-ad2a-b041a4bd3119 admin admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:25.415434 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 742/1482] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:20:24 2022] DELETE /identity/v3/users/c24aa3f7f951483d991e59c93f01ce49 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 00:20:25.417276 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9dda84-6969-4385-ad2a-b041a4bd3119 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:25.421838 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47169bc3-34d8-4005-80d1-b50f70b34429 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:25.424393 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47169bc3-34d8-4005-80d1-b50f70b34429 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:25.425967 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47169bc3-34d8-4005-80d1-b50f70b34429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:25.427172 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9dda84-6969-4385-ad2a-b041a4bd3119 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:25.522190 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-47169bc3-34d8-4005-80d1-b50f70b34429 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:25.533622 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-47169bc3-34d8-4005-80d1-b50f70b34429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:25.618473 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 741/1483] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:20:25 2022] POST /identity/v3/users => generated 284 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:20:25.626824 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9dda84-6969-4385-ad2a-b041a4bd3119 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:25.630432 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-1c9dda84-6969-4385-ad2a-b041a4bd3119 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:25.680440 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9dda84-6969-4385-ad2a-b041a4bd3119 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:25.735268 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 743/1484] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:25.830346 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073a38c5-a2f2-4590-bd55-39bff05281a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:25.831459 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073a38c5-a2f2-4590-bd55-39bff05281a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:25.831848 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073a38c5-a2f2-4590-bd55-39bff05281a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:25.832276 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073a38c5-a2f2-4590-bd55-39bff05281a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:25.833487 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073a38c5-a2f2-4590-bd55-39bff05281a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:25.835766 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073a38c5-a2f2-4590-bd55-39bff05281a1 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:25.846727 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47169bc3-34d8-4005-80d1-b50f70b34429 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:25.853872 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-47169bc3-34d8-4005-80d1-b50f70b34429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:25.898824 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 742/1485] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:20:25 2022] GET /identity/v3/roles => generated 1420 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:20:25.902111 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073a38c5-a2f2-4590-bd55-39bff05281a1 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:25.956762 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073a38c5-a2f2-4590-bd55-39bff05281a1 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:25.981639 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72795d2a-97c2-43da-8eb0-da29904b21be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:25.983046 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72795d2a-97c2-43da-8eb0-da29904b21be admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:25.983597 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72795d2a-97c2-43da-8eb0-da29904b21be admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:25.989748 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72795d2a-97c2-43da-8eb0-da29904b21be admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:25.989748 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72795d2a-97c2-43da-8eb0-da29904b21be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:25.989748 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72795d2a-97c2-43da-8eb0-da29904b21be admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:26.019851 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 743/1486] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:20:25 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:20:26.021316 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72795d2a-97c2-43da-8eb0-da29904b21be admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:26.026093 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47169bc3-34d8-4005-80d1-b50f70b34429 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:26.039074 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d745949f-9cc9-4a69-8957-11c43f99924a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:26.047409 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d745949f-9cc9-4a69-8957-11c43f99924a service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:26.047409 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d745949f-9cc9-4a69-8957-11c43f99924a service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:26.047409 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d745949f-9cc9-4a69-8957-11c43f99924a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:26.048554 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72795d2a-97c2-43da-8eb0-da29904b21be admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:26.055062 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494bb62c-366f-49dc-b16f-eff2a8a4a48e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:26.056006 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494bb62c-366f-49dc-b16f-eff2a8a4a48e admin admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:26.056462 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494bb62c-366f-49dc-b16f-eff2a8a4a48e admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:26.057387 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494bb62c-366f-49dc-b16f-eff2a8a4a48e admin admin] PATH_INFO: `/v3/projects/093c7ca07d6d4ae089e9c1f787d2e06c/users/6b79888e4a7042118ec0f66692a9b549/roles/ef689f96442945e8a7bc8cbfee13d909` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:26.064545 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d745949f-9cc9-4a69-8957-11c43f99924a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:26.065879 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d745949f-9cc9-4a69-8957-11c43f99924a service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:26.080609 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494bb62c-366f-49dc-b16f-eff2a8a4a48e admin admin] RBAC: Authorizing `identity:create_grant(project_id=093c7ca07d6d4ae089e9c1f787d2e06c, user_id=6b79888e4a7042118ec0f66692a9b549, role_id=ef689f96442945e8a7bc8cbfee13d909)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:26.084403 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494bb62c-366f-49dc-b16f-eff2a8a4a48e admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:26.091185 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 744/1487] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:20:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:20:26.188755 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 744/1488] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:20:26 2022] PUT /identity/v3/projects/093c7ca07d6d4ae089e9c1f787d2e06c/users/6b79888e4a7042118ec0f66692a9b549/roles/ef689f96442945e8a7bc8cbfee13d909 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:26.202410 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddf7d33-3cb2-4356-9422-74566288b392 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:26.202804 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddf7d33-3cb2-4356-9422-74566288b392 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:26.202981 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddf7d33-3cb2-4356-9422-74566288b392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:26.297115 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-fddf7d33-3cb2-4356-9422-74566288b392 None None] MFA Rules not processed for user `6b79888e4a7042118ec0f66692a9b549`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:26.303836 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-fddf7d33-3cb2-4356-9422-74566288b392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:26.328614 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:26.466113 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 745/1489] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:26 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:26.472641 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494bb62c-366f-49dc-b16f-eff2a8a4a48e admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:26.638331 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494bb62c-366f-49dc-b16f-eff2a8a4a48e admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:26.645709 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323f4790-3c87-425a-ade6-69579e0d4b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:26.646399 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323f4790-3c87-425a-ade6-69579e0d4b39 admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:26.646751 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323f4790-3c87-425a-ade6-69579e0d4b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:26.647080 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323f4790-3c87-425a-ade6-69579e0d4b39 admin admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:26.647867 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323f4790-3c87-425a-ade6-69579e0d4b39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:26.650501 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323f4790-3c87-425a-ade6-69579e0d4b39 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:26.749398 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:26.766902 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 745/1490] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:20:26 2022] POST /identity/v3/projects => generated 367 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:20:26.768187 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddf7d33-3cb2-4356-9422-74566288b392 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:26.788673 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddf7d33-3cb2-4356-9422-74566288b392 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:26.818373 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c996ecb2-3713-454a-ae5c-a929a88259af None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:26.818713 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c996ecb2-3713-454a-ae5c-a929a88259af None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:26.819039 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c996ecb2-3713-454a-ae5c-a929a88259af None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:26.821092 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 746/1491] 10.222.0.36 () {54 vars in 921 bytes} [Wed Oct 12 00:20:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 00:20:26.856813 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1ce292-40eb-4f1a-9016-b0feaa0b6e3b None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:26.857169 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1ce292-40eb-4f1a-9016-b0feaa0b6e3b None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:26.857393 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1ce292-40eb-4f1a-9016-b0feaa0b6e3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:26.937370 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-8e1ce292-40eb-4f1a-9016-b0feaa0b6e3b None None] MFA Rules not processed for user `a515b75aa3944912ad2364ffd7924f40`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:26.942945 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-8e1ce292-40eb-4f1a-9016-b0feaa0b6e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:26.974951 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844563c9-dd8c-4638-b451-6524c3bb8ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:26.977521 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844563c9-dd8c-4638-b451-6524c3bb8ad1 admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:26.979245 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844563c9-dd8c-4638-b451-6524c3bb8ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:26.979643 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844563c9-dd8c-4638-b451-6524c3bb8ad1 admin admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:26.981434 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844563c9-dd8c-4638-b451-6524c3bb8ad1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:26.986453 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844563c9-dd8c-4638-b451-6524c3bb8ad1 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:27.075535 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 746/1492] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:20:26 2022] POST /identity/v3/users => generated 290 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:20:27.077727 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844563c9-dd8c-4638-b451-6524c3bb8ad1 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:27.081945 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-844563c9-dd8c-4638-b451-6524c3bb8ad1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:27.121150 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844563c9-dd8c-4638-b451-6524c3bb8ad1 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:27.136432 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebce6c6-dd37-4eef-987a-f6fe0283e29c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:27.137244 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebce6c6-dd37-4eef-987a-f6fe0283e29c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:27.139516 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebce6c6-dd37-4eef-987a-f6fe0283e29c admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:27.140244 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebce6c6-dd37-4eef-987a-f6fe0283e29c admin admin] PATH_INFO: `/v3/users/716cc08f22ec4f36931f3fda7a137a58` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:27.143504 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebce6c6-dd37-4eef-987a-f6fe0283e29c admin admin] RBAC: Authorizing `identity:delete_user(user_id=716cc08f22ec4f36931f3fda7a137a58)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:27.150697 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebce6c6-dd37-4eef-987a-f6fe0283e29c admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:27.179233 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 747/1493] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Oct 12 00:20:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 00:20:27.182005 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1ce292-40eb-4f1a-9016-b0feaa0b6e3b None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:27.208933 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1ce292-40eb-4f1a-9016-b0feaa0b6e3b None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:27.226652 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4beff79-3293-4dcd-92a8-1c46b099ef0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:27.227946 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4beff79-3293-4dcd-92a8-1c46b099ef0b admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:27.228731 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4beff79-3293-4dcd-92a8-1c46b099ef0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:27.229443 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4beff79-3293-4dcd-92a8-1c46b099ef0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:27.230431 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4beff79-3293-4dcd-92a8-1c46b099ef0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:27.232739 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4beff79-3293-4dcd-92a8-1c46b099ef0b admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:27.259931 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 748/1494] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:20:27 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:20:27.264504 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4beff79-3293-4dcd-92a8-1c46b099ef0b admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:27.268501 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-a4beff79-3293-4dcd-92a8-1c46b099ef0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:27.300787 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4beff79-3293-4dcd-92a8-1c46b099ef0b admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:27.320224 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdddb6f6-83ec-4ecf-9a33-02f6bb0dc079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:27.321639 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdddb6f6-83ec-4ecf-9a33-02f6bb0dc079 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:27.324667 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdddb6f6-83ec-4ecf-9a33-02f6bb0dc079 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:27.330900 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdddb6f6-83ec-4ecf-9a33-02f6bb0dc079 admin admin] PATH_INFO: `/v3/users/4796dae1f12f4163b0d0776e5fdd7903` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:27.330900 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdddb6f6-83ec-4ecf-9a33-02f6bb0dc079 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4796dae1f12f4163b0d0776e5fdd7903)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:27.333809 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdddb6f6-83ec-4ecf-9a33-02f6bb0dc079 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:27.381932 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-aebce6c6-dd37-4eef-987a-f6fe0283e29c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '716cc08f22ec4f36931f3fda7a137a58'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:27.383955 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-aebce6c6-dd37-4eef-987a-f6fe0283e29c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '716cc08f22ec4f36931f3fda7a137a58'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:27.413980 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-aebce6c6-dd37-4eef-987a-f6fe0283e29c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '716cc08f22ec4f36931f3fda7a137a58'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:27.541209 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-aebce6c6-dd37-4eef-987a-f6fe0283e29c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '716cc08f22ec4f36931f3fda7a137a58'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:27.542422 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-aebce6c6-dd37-4eef-987a-f6fe0283e29c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '716cc08f22ec4f36931f3fda7a137a58'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:27.576340 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 747/1495] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:20:27 2022] DELETE /identity/v3/users/716cc08f22ec4f36931f3fda7a137a58 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:27.576340 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebce6c6-dd37-4eef-987a-f6fe0283e29c admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:27.576340 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-aebce6c6-dd37-4eef-987a-f6fe0283e29c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:27.631780 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-cdddb6f6-83ec-4ecf-9a33-02f6bb0dc079 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4796dae1f12f4163b0d0776e5fdd7903'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:27.633659 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-cdddb6f6-83ec-4ecf-9a33-02f6bb0dc079 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4796dae1f12f4163b0d0776e5fdd7903'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:27.679958 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-cdddb6f6-83ec-4ecf-9a33-02f6bb0dc079 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4796dae1f12f4163b0d0776e5fdd7903'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:27.692175 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:27.775039 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebce6c6-dd37-4eef-987a-f6fe0283e29c admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:27.777238 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-cdddb6f6-83ec-4ecf-9a33-02f6bb0dc079 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4796dae1f12f4163b0d0776e5fdd7903'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:27.777937 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-cdddb6f6-83ec-4ecf-9a33-02f6bb0dc079 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4796dae1f12f4163b0d0776e5fdd7903'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:27.791905 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59565a9b-9a13-4ca5-84f7-0c9e5a05c9fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:27.794219 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59565a9b-9a13-4ca5-84f7-0c9e5a05c9fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:27.794219 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59565a9b-9a13-4ca5-84f7-0c9e5a05c9fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:27.794570 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59565a9b-9a13-4ca5-84f7-0c9e5a05c9fe admin admin] PATH_INFO: `/v3/projects/ae7c15f10b244e03a6a2fb3cfc73b24d/users/ac05a769b7fb41a3a0d1f756b0f54e0e/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:27.803045 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 749/1496] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:20:27 2022] DELETE /identity/v3/users/4796dae1f12f4163b0d0776e5fdd7903 => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:27.803045 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6b043c-9de1-485c-b8e4-792bdd26d77f None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:27.803045 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6b043c-9de1-485c-b8e4-792bdd26d77f None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:27.803045 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6b043c-9de1-485c-b8e4-792bdd26d77f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:27.847061 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-fe6b043c-9de1-485c-b8e4-792bdd26d77f None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:27.855305 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-fe6b043c-9de1-485c-b8e4-792bdd26d77f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:27.867700 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59565a9b-9a13-4ca5-84f7-0c9e5a05c9fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae7c15f10b244e03a6a2fb3cfc73b24d, user_id=ac05a769b7fb41a3a0d1f756b0f54e0e, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:27.871289 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59565a9b-9a13-4ca5-84f7-0c9e5a05c9fe admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:27.935697 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:27.966470 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 750/1497] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:27.968123 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa8234-ced4-4291-9b10-9eb7543e481c None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:27.968409 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa8234-ced4-4291-9b10-9eb7543e481c None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:27.968900 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa8234-ced4-4291-9b10-9eb7543e481c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:27.969014 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 748/1498] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:20:27 2022] PUT /identity/v3/projects/ae7c15f10b244e03a6a2fb3cfc73b24d/users/ac05a769b7fb41a3a0d1f756b0f54e0e/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:27.970415 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59565a9b-9a13-4ca5-84f7-0c9e5a05c9fe admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:27.972080 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-59565a9b-9a13-4ca5-84f7-0c9e5a05c9fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:27.994103 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-67aa8234-ced4-4291-9b10-9eb7543e481c None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:27.995342 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-67aa8234-ced4-4291-9b10-9eb7543e481c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:28.118094 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 751/1499] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:28.119865 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aa8234-ced4-4291-9b10-9eb7543e481c None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:28.121420 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-67aa8234-ced4-4291-9b10-9eb7543e481c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:28.140004 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aa8234-ced4-4291-9b10-9eb7543e481c None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:28.148086 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b2bef5-e491-4c18-ad7c-28dfa4dc7d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:28.148743 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b2bef5-e491-4c18-ad7c-28dfa4dc7d1e admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:28.149029 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b2bef5-e491-4c18-ad7c-28dfa4dc7d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:28.149404 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b2bef5-e491-4c18-ad7c-28dfa4dc7d1e admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:28.150069 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b2bef5-e491-4c18-ad7c-28dfa4dc7d1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:28.161647 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b2bef5-e491-4c18-ad7c-28dfa4dc7d1e admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:28.164789 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59565a9b-9a13-4ca5-84f7-0c9e5a05c9fe admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:28.179690 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 752/1500] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:20:28 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:20:28.183440 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff164f4e-85ce-4ac0-9cb8-d26c69620f9e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a515b75aa3944912ad2364ffd7924f40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:28.183440 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff164f4e-85ce-4ac0-9cb8-d26c69620f9e service nova] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:28.183440 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff164f4e-85ce-4ac0-9cb8-d26c69620f9e service nova] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:28.183440 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff164f4e-85ce-4ac0-9cb8-d26c69620f9e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:28.184943 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b2bef5-e491-4c18-ad7c-28dfa4dc7d1e admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:28.188105 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-86b2bef5-e491-4c18-ad7c-28dfa4dc7d1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:28.192974 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-ff164f4e-85ce-4ac0-9cb8-d26c69620f9e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:28.226988 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff164f4e-85ce-4ac0-9cb8-d26c69620f9e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:28.229453 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff164f4e-85ce-4ac0-9cb8-d26c69620f9e service nova] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:28.255219 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 749/1501] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:20:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:20:28.270325 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff164f4e-85ce-4ac0-9cb8-d26c69620f9e service nova] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:28.277287 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b2bef5-e491-4c18-ad7c-28dfa4dc7d1e admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:28.281584 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19a6c88-e0d5-4417-9c44-abb01bee18ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:28.286688 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19a6c88-e0d5-4417-9c44-abb01bee18ab service neutron] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:28.286688 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19a6c88-e0d5-4417-9c44-abb01bee18ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:28.286688 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19a6c88-e0d5-4417-9c44-abb01bee18ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:28.297053 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff164f4e-85ce-4ac0-9cb8-d26c69620f9e service nova] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:28.305380 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19a6c88-e0d5-4417-9c44-abb01bee18ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:28.306516 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28c199e-dfa1-4d4c-bd7c-15ba8131554c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:28.307136 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28c199e-dfa1-4d4c-bd7c-15ba8131554c service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:28.307452 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28c199e-dfa1-4d4c-bd7c-15ba8131554c service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:28.307728 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28c199e-dfa1-4d4c-bd7c-15ba8131554c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:28.309421 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19a6c88-e0d5-4417-9c44-abb01bee18ab service neutron] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:28.326491 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28c199e-dfa1-4d4c-bd7c-15ba8131554c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:28.328082 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28c199e-dfa1-4d4c-bd7c-15ba8131554c service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:28.347515 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 753/1502] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:20:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:20:28.350285 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19a6c88-e0d5-4417-9c44-abb01bee18ab service neutron] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:28.350633 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 750/1503] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:20:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:20:28.374814 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19a6c88-e0d5-4417-9c44-abb01bee18ab service neutron] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:28.410606 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d7fbe2-2eea-4608-8fcb-fc9731589853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:28.411284 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d7fbe2-2eea-4608-8fcb-fc9731589853 admin admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:28.420262 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d7fbe2-2eea-4608-8fcb-fc9731589853 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:28.420580 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d7fbe2-2eea-4608-8fcb-fc9731589853 admin admin] PATH_INFO: `/v3/projects/ae7c15f10b244e03a6a2fb3cfc73b24d/users/ac05a769b7fb41a3a0d1f756b0f54e0e/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:28.428236 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d7fbe2-2eea-4608-8fcb-fc9731589853 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae7c15f10b244e03a6a2fb3cfc73b24d, user_id=ac05a769b7fb41a3a0d1f756b0f54e0e, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:28.431330 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d7fbe2-2eea-4608-8fcb-fc9731589853 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:28.481310 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 754/1504] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:20:28 2022] PUT /identity/v3/projects/ae7c15f10b244e03a6a2fb3cfc73b24d/users/ac05a769b7fb41a3a0d1f756b0f54e0e/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:28.483891 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046eefc2-4c40-45d7-b79a-8a9d5cdf6ce0 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:28.484223 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046eefc2-4c40-45d7-b79a-8a9d5cdf6ce0 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:28.484506 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046eefc2-4c40-45d7-b79a-8a9d5cdf6ce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:28.519811 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-046eefc2-4c40-45d7-b79a-8a9d5cdf6ce0 None None] MFA Rules not processed for user `ac05a769b7fb41a3a0d1f756b0f54e0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:28.521721 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-046eefc2-4c40-45d7-b79a-8a9d5cdf6ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:28.658291 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 751/1505] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:28 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:28.668854 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d7fbe2-2eea-4608-8fcb-fc9731589853 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:28.673609 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-45d7fbe2-2eea-4608-8fcb-fc9731589853 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:28.763247 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:28.814512 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d7fbe2-2eea-4608-8fcb-fc9731589853 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:28.821881 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3c73ce-df8c-4267-abc2-652a7d34a282 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:28.827565 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3c73ce-df8c-4267-abc2-652a7d34a282 service cinder] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:28.827565 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3c73ce-df8c-4267-abc2-652a7d34a282 service cinder] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:28.827565 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3c73ce-df8c-4267-abc2-652a7d34a282 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:28.837403 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3c73ce-df8c-4267-abc2-652a7d34a282 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:28.840390 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3c73ce-df8c-4267-abc2-652a7d34a282 service cinder] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:28.873236 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 755/1506] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:20:28 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:20:28.903135 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:28.957206 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b534767-43db-45f9-a752-05904bd7f34b None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:28.957568 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b534767-43db-45f9-a752-05904bd7f34b None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:28.957954 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b534767-43db-45f9-a752-05904bd7f34b None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:28.959827 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 752/1507] 10.222.0.36 () {54 vars in 921 bytes} [Wed Oct 12 00:20:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:20:29.005459 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3c73ce-df8c-4267-abc2-652a7d34a282 service cinder] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:29.008179 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-3f3c73ce-df8c-4267-abc2-652a7d34a282 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:29.024838 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3c73ce-df8c-4267-abc2-652a7d34a282 service cinder] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:29.031618 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4dc936-23a1-4af3-a2a7-9927fac145af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:29.033200 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4dc936-23a1-4af3-a2a7-9927fac145af service cinder] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:29.034054 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4dc936-23a1-4af3-a2a7-9927fac145af service cinder] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:29.034798 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4dc936-23a1-4af3-a2a7-9927fac145af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:29.038231 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-ad4dc936-23a1-4af3-a2a7-9927fac145af service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:29.073257 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ce4227-12c4-4aab-9473-49d515ff5a4d None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:29.073924 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ce4227-12c4-4aab-9473-49d515ff5a4d None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:29.076564 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ce4227-12c4-4aab-9473-49d515ff5a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:29.121075 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-e4ce4227-12c4-4aab-9473-49d515ff5a4d None None] MFA Rules not processed for user `a515b75aa3944912ad2364ffd7924f40`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:29.123092 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-e4ce4227-12c4-4aab-9473-49d515ff5a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:29.183777 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4dc936-23a1-4af3-a2a7-9927fac145af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:29.186736 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4dc936-23a1-4af3-a2a7-9927fac145af service cinder] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:29.217823 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 756/1508] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:20:29 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:20:29.300035 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 753/1509] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Oct 12 00:20:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:29.628101 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4dc936-23a1-4af3-a2a7-9927fac145af service cinder] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:29.655878 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4dc936-23a1-4af3-a2a7-9927fac145af service cinder] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:29.682277 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a968b58b-a900-44ac-b946-7967ee28fb00 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a515b75aa3944912ad2364ffd7924f40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:29.682995 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a968b58b-a900-44ac-b946-7967ee28fb00 service nova] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:29.683404 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a968b58b-a900-44ac-b946-7967ee28fb00 service nova] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:29.684856 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a968b58b-a900-44ac-b946-7967ee28fb00 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:29.701874 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ce4227-12c4-4aab-9473-49d515ff5a4d None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:29.703762 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-e4ce4227-12c4-4aab-9473-49d515ff5a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:29.715317 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a968b58b-a900-44ac-b946-7967ee28fb00 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:29.723640 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a968b58b-a900-44ac-b946-7967ee28fb00 service nova] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:29.762952 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 757/1510] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:20:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 00:20:29.917516 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ce4227-12c4-4aab-9473-49d515ff5a4d None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:29.927032 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d709ef7e-079a-45f3-8e21-87a3356a940a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:29.927751 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d709ef7e-079a-45f3-8e21-87a3356a940a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:29.928079 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d709ef7e-079a-45f3-8e21-87a3356a940a admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:29.928304 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d709ef7e-079a-45f3-8e21-87a3356a940a admin admin] PATH_INFO: `/v3/projects/a14e5c00a0ff4333b5931205f19bfe91` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:29.930270 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d709ef7e-079a-45f3-8e21-87a3356a940a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a14e5c00a0ff4333b5931205f19bfe91)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:29.932618 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d709ef7e-079a-45f3-8e21-87a3356a940a admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:29.967126 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:30.120101 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-d709ef7e-079a-45f3-8e21-87a3356a940a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a14e5c00a0ff4333b5931205f19bfe91', 'request_id': 'req-d709ef7e-079a-45f3-8e21-87a3356a940a'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:30.139340 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-d709ef7e-079a-45f3-8e21-87a3356a940a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a14e5c00a0ff4333b5931205f19bfe91', 'request_id': 'req-d709ef7e-079a-45f3-8e21-87a3356a940a'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:30.240570 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-d709ef7e-079a-45f3-8e21-87a3356a940a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a14e5c00a0ff4333b5931205f19bfe91', 'request_id': 'req-d709ef7e-079a-45f3-8e21-87a3356a940a'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:30.258927 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-d709ef7e-079a-45f3-8e21-87a3356a940a admin admin] The token cache is being invalidate because project a14e5c00a0ff4333b5931205f19bfe91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81746) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:20:30.259224 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-d709ef7e-079a-45f3-8e21-87a3356a940a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:30.261664 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-d709ef7e-079a-45f3-8e21-87a3356a940a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:30.264831 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 754/1511] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:20:29 2022] DELETE /identity/v3/projects/a14e5c00a0ff4333b5931205f19bfe91 => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:30.268869 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a968b58b-a900-44ac-b946-7967ee28fb00 service nova] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:30.293362 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a968b58b-a900-44ac-b946-7967ee28fb00 service nova] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:30.391651 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:30.481409 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57a841b-d2dd-4252-a99f-4e954af06ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:30.481409 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57a841b-d2dd-4252-a99f-4e954af06ba3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:30.481409 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57a841b-d2dd-4252-a99f-4e954af06ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:30.481409 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57a841b-d2dd-4252-a99f-4e954af06ba3 admin admin] PATH_INFO: `/v3/users/099caafb51b742028bb978fb6fb43c09` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:30.483047 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57a841b-d2dd-4252-a99f-4e954af06ba3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=099caafb51b742028bb978fb6fb43c09)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:30.488410 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57a841b-d2dd-4252-a99f-4e954af06ba3 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:30.750652 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-b57a841b-d2dd-4252-a99f-4e954af06ba3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '099caafb51b742028bb978fb6fb43c09'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:30.751705 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-b57a841b-d2dd-4252-a99f-4e954af06ba3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '099caafb51b742028bb978fb6fb43c09'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:30.768923 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-b57a841b-d2dd-4252-a99f-4e954af06ba3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '099caafb51b742028bb978fb6fb43c09'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:30.863051 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-b57a841b-d2dd-4252-a99f-4e954af06ba3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '099caafb51b742028bb978fb6fb43c09'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:30.864465 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-b57a841b-d2dd-4252-a99f-4e954af06ba3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '099caafb51b742028bb978fb6fb43c09'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:30.885824 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 758/1512] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:20:30 2022] DELETE /identity/v3/users/099caafb51b742028bb978fb6fb43c09 => generated 0 bytes in 614 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:31.134044 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:31.573198 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d709ef7e-079a-45f3-8e21-87a3356a940a admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:31.576075 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-d709ef7e-079a-45f3-8e21-87a3356a940a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:31.621666 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d709ef7e-079a-45f3-8e21-87a3356a940a admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:31.782465 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e05321-c576-4c01-bde4-5ce8c0df8571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:31.783393 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e05321-c576-4c01-bde4-5ce8c0df8571 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:31.789482 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e05321-c576-4c01-bde4-5ce8c0df8571 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:31.789482 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e05321-c576-4c01-bde4-5ce8c0df8571 admin admin] PATH_INFO: `/v3/projects/9dd2567c3d2642e7ae57372262edd1f9` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:31.789482 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e05321-c576-4c01-bde4-5ce8c0df8571 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9dd2567c3d2642e7ae57372262edd1f9)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:31.789482 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e05321-c576-4c01-bde4-5ce8c0df8571 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:32.013886 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-e1e05321-c576-4c01-bde4-5ce8c0df8571 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9dd2567c3d2642e7ae57372262edd1f9', 'request_id': 'req-e1e05321-c576-4c01-bde4-5ce8c0df8571'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:32.022164 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-e1e05321-c576-4c01-bde4-5ce8c0df8571 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9dd2567c3d2642e7ae57372262edd1f9', 'request_id': 'req-e1e05321-c576-4c01-bde4-5ce8c0df8571'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:32.055460 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57a841b-d2dd-4252-a99f-4e954af06ba3 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:32.057545 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-b57a841b-d2dd-4252-a99f-4e954af06ba3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:32.099990 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57a841b-d2dd-4252-a99f-4e954af06ba3 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:32.124997 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-e1e05321-c576-4c01-bde4-5ce8c0df8571 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9dd2567c3d2642e7ae57372262edd1f9', 'request_id': 'req-e1e05321-c576-4c01-bde4-5ce8c0df8571'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:32.153104 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-e1e05321-c576-4c01-bde4-5ce8c0df8571 admin admin] The token cache is being invalidate because project 9dd2567c3d2642e7ae57372262edd1f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81746) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:20:32.166064 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-e1e05321-c576-4c01-bde4-5ce8c0df8571 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:32.166064 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-e1e05321-c576-4c01-bde4-5ce8c0df8571 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:32.166064 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 755/1513] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:20:31 2022] DELETE /identity/v3/projects/9dd2567c3d2642e7ae57372262edd1f9 => generated 0 bytes in 585 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:32.166064 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e05321-c576-4c01-bde4-5ce8c0df8571 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:32.166064 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-e1e05321-c576-4c01-bde4-5ce8c0df8571 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:32.296791 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e05321-c576-4c01-bde4-5ce8c0df8571 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:32.303028 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c84cbf-6b12-47b0-8664-b7e2a4d371d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:32.304137 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c84cbf-6b12-47b0-8664-b7e2a4d371d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:32.304616 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c84cbf-6b12-47b0-8664-b7e2a4d371d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:32.305083 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c84cbf-6b12-47b0-8664-b7e2a4d371d4 admin admin] PATH_INFO: `/v3/users/2472c5efc0c1408da76d30191b5f7496` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:32.306543 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c84cbf-6b12-47b0-8664-b7e2a4d371d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2472c5efc0c1408da76d30191b5f7496)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:32.308757 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c84cbf-6b12-47b0-8664-b7e2a4d371d4 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:32.313036 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b070e09-0174-4394-85f3-f0bc92a592fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:32.316742 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b070e09-0174-4394-85f3-f0bc92a592fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:32.316861 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b070e09-0174-4394-85f3-f0bc92a592fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:32.317357 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b070e09-0174-4394-85f3-f0bc92a592fd admin admin] PATH_INFO: `/v3/projects/1761e198d6d44288807d407cdcec48c5` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:32.331412 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b070e09-0174-4394-85f3-f0bc92a592fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=1761e198d6d44288807d407cdcec48c5)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:32.338810 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b070e09-0174-4394-85f3-f0bc92a592fd admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:32.540653 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-99c84cbf-6b12-47b0-8664-b7e2a4d371d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2472c5efc0c1408da76d30191b5f7496'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:32.542353 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-99c84cbf-6b12-47b0-8664-b7e2a4d371d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2472c5efc0c1408da76d30191b5f7496'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:32.559431 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-6b070e09-0174-4394-85f3-f0bc92a592fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1761e198d6d44288807d407cdcec48c5', 'request_id': 'req-6b070e09-0174-4394-85f3-f0bc92a592fd'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:32.568267 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-99c84cbf-6b12-47b0-8664-b7e2a4d371d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2472c5efc0c1408da76d30191b5f7496'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:32.574301 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-6b070e09-0174-4394-85f3-f0bc92a592fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1761e198d6d44288807d407cdcec48c5', 'request_id': 'req-6b070e09-0174-4394-85f3-f0bc92a592fd'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:32.711473 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-6b070e09-0174-4394-85f3-f0bc92a592fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1761e198d6d44288807d407cdcec48c5', 'request_id': 'req-6b070e09-0174-4394-85f3-f0bc92a592fd'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:32.734025 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-6b070e09-0174-4394-85f3-f0bc92a592fd admin admin] The token cache is being invalidate because project 1761e198d6d44288807d407cdcec48c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81747) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:20:32.738840 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-6b070e09-0174-4394-85f3-f0bc92a592fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:32.738840 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-6b070e09-0174-4394-85f3-f0bc92a592fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:32.746136 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 759/1514] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:20:32 2022] DELETE /identity/v3/projects/1761e198d6d44288807d407cdcec48c5 => generated 0 bytes in 689 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 00:20:32.751349 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b070e09-0174-4394-85f3-f0bc92a592fd admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:32.759938 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:32.773519 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-6b070e09-0174-4394-85f3-f0bc92a592fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:32.825043 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-99c84cbf-6b12-47b0-8664-b7e2a4d371d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2472c5efc0c1408da76d30191b5f7496'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:32.825739 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-99c84cbf-6b12-47b0-8664-b7e2a4d371d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2472c5efc0c1408da76d30191b5f7496'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:32.859196 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 756/1515] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:20:32 2022] DELETE /identity/v3/users/2472c5efc0c1408da76d30191b5f7496 => generated 0 bytes in 698 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:33.058337 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b070e09-0174-4394-85f3-f0bc92a592fd admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:33.193928 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eda7997-866c-4521-85c9-dfa315835935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:33.194592 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eda7997-866c-4521-85c9-dfa315835935 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:33.194887 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eda7997-866c-4521-85c9-dfa315835935 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:33.195162 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eda7997-866c-4521-85c9-dfa315835935 admin admin] PATH_INFO: `/v3/users/0da7140e0e70415b9e6b453ec71546da` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:33.196611 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eda7997-866c-4521-85c9-dfa315835935 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0da7140e0e70415b9e6b453ec71546da)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:33.199105 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eda7997-866c-4521-85c9-dfa315835935 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:33.498109 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-4eda7997-866c-4521-85c9-dfa315835935 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0da7140e0e70415b9e6b453ec71546da'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:33.499796 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-4eda7997-866c-4521-85c9-dfa315835935 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0da7140e0e70415b9e6b453ec71546da'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:33.566028 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-4eda7997-866c-4521-85c9-dfa315835935 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0da7140e0e70415b9e6b453ec71546da'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:33.659397 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-4eda7997-866c-4521-85c9-dfa315835935 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0da7140e0e70415b9e6b453ec71546da'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:33.659397 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-4eda7997-866c-4521-85c9-dfa315835935 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0da7140e0e70415b9e6b453ec71546da'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:33.667468 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c84cbf-6b12-47b0-8664-b7e2a4d371d4 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:33.667832 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-99c84cbf-6b12-47b0-8664-b7e2a4d371d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:33.679906 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 760/1516] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:20:32 2022] DELETE /identity/v3/users/0da7140e0e70415b9e6b453ec71546da => generated 0 bytes in 931 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 00:20:33.698753 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c84cbf-6b12-47b0-8664-b7e2a4d371d4 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:33.853905 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b9a059-596b-48a0-ba1e-f36606d75381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:33.854767 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b9a059-596b-48a0-ba1e-f36606d75381 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:33.855453 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b9a059-596b-48a0-ba1e-f36606d75381 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:33.855544 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b9a059-596b-48a0-ba1e-f36606d75381 admin admin] PATH_INFO: `/v3/projects/ba0e3f466bee4bbf8a92a23a2ccdb80a` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:33.861125 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b9a059-596b-48a0-ba1e-f36606d75381 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba0e3f466bee4bbf8a92a23a2ccdb80a)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:33.862146 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eda7997-866c-4521-85c9-dfa315835935 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:33.864542 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-4eda7997-866c-4521-85c9-dfa315835935 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:33.870177 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b9a059-596b-48a0-ba1e-f36606d75381 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:33.898142 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eda7997-866c-4521-85c9-dfa315835935 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:33.914564 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52760989-2eb4-46a4-bc8a-b753a6bfb048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:33.915234 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52760989-2eb4-46a4-bc8a-b753a6bfb048 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:33.919395 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52760989-2eb4-46a4-bc8a-b753a6bfb048 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:33.920403 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52760989-2eb4-46a4-bc8a-b753a6bfb048 admin admin] PATH_INFO: `/v3/projects/58a4724798604fd789c15159e5e79ea0` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:33.927707 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52760989-2eb4-46a4-bc8a-b753a6bfb048 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58a4724798604fd789c15159e5e79ea0)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:33.936561 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52760989-2eb4-46a4-bc8a-b753a6bfb048 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:34.016881 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-13b9a059-596b-48a0-ba1e-f36606d75381 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba0e3f466bee4bbf8a92a23a2ccdb80a', 'request_id': 'req-13b9a059-596b-48a0-ba1e-f36606d75381'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:34.039082 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-13b9a059-596b-48a0-ba1e-f36606d75381 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba0e3f466bee4bbf8a92a23a2ccdb80a', 'request_id': 'req-13b9a059-596b-48a0-ba1e-f36606d75381'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:34.185136 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-13b9a059-596b-48a0-ba1e-f36606d75381 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba0e3f466bee4bbf8a92a23a2ccdb80a', 'request_id': 'req-13b9a059-596b-48a0-ba1e-f36606d75381'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:34.190099 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-52760989-2eb4-46a4-bc8a-b753a6bfb048 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58a4724798604fd789c15159e5e79ea0', 'request_id': 'req-52760989-2eb4-46a4-bc8a-b753a6bfb048'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:34.209928 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-52760989-2eb4-46a4-bc8a-b753a6bfb048 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58a4724798604fd789c15159e5e79ea0', 'request_id': 'req-52760989-2eb4-46a4-bc8a-b753a6bfb048'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:34.221497 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-13b9a059-596b-48a0-ba1e-f36606d75381 admin admin] The token cache is being invalidate because project ba0e3f466bee4bbf8a92a23a2ccdb80a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81746) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:20:34.222412 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-13b9a059-596b-48a0-ba1e-f36606d75381 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:34.223499 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-13b9a059-596b-48a0-ba1e-f36606d75381 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:34.228432 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 757/1517] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:20:33 2022] DELETE /identity/v3/projects/ba0e3f466bee4bbf8a92a23a2ccdb80a => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 00:20:34.233532 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9816f04e-c790-4e2b-b611-a5130b2b915c None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:34.233884 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9816f04e-c790-4e2b-b611-a5130b2b915c None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:34.234374 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9816f04e-c790-4e2b-b611-a5130b2b915c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:34.266262 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-9816f04e-c790-4e2b-b611-a5130b2b915c None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:34.267754 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-9816f04e-c790-4e2b-b611-a5130b2b915c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:34.271980 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-52760989-2eb4-46a4-bc8a-b753a6bfb048 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58a4724798604fd789c15159e5e79ea0', 'request_id': 'req-52760989-2eb4-46a4-bc8a-b753a6bfb048'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:34.273687 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:34.290147 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-52760989-2eb4-46a4-bc8a-b753a6bfb048 admin admin] The token cache is being invalidate because project 58a4724798604fd789c15159e5e79ea0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81747) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:20:34.290439 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-52760989-2eb4-46a4-bc8a-b753a6bfb048 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:34.291746 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-52760989-2eb4-46a4-bc8a-b753a6bfb048 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:34.299536 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 761/1518] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:20:33 2022] DELETE /identity/v3/projects/58a4724798604fd789c15159e5e79ea0 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:34.302882 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9f52c9-f6a4-4138-85c1-43d862c73210 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:34.303521 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9f52c9-f6a4-4138-85c1-43d862c73210 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:34.304622 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9f52c9-f6a4-4138-85c1-43d862c73210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:34.363606 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-ca9f52c9-f6a4-4138-85c1-43d862c73210 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:34.365106 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-ca9f52c9-f6a4-4138-85c1-43d862c73210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:34.382049 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 758/1519] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:34.396927 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f75b15-03ce-4556-938d-18c8a97b68d7 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:34.397232 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f75b15-03ce-4556-938d-18c8a97b68d7 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:34.397487 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f75b15-03ce-4556-938d-18c8a97b68d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:34.460444 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-a0f75b15-03ce-4556-938d-18c8a97b68d7 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:34.469337 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-a0f75b15-03ce-4556-938d-18c8a97b68d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:34.482872 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 759/1520] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:34.486896 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f75b15-03ce-4556-938d-18c8a97b68d7 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:34.510695 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f75b15-03ce-4556-938d-18c8a97b68d7 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:34.518120 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 762/1521] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:34.525736 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92ead9f-9d52-4c24-89c4-b730b2947901 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:34.526025 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92ead9f-9d52-4c24-89c4-b730b2947901 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:34.526301 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92ead9f-9d52-4c24-89c4-b730b2947901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:34.532384 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da49ff89-4504-448e-a953-3740111339fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:34.532384 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da49ff89-4504-448e-a953-3740111339fb admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:34.532384 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da49ff89-4504-448e-a953-3740111339fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:34.532384 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da49ff89-4504-448e-a953-3740111339fb admin admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:34.536323 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da49ff89-4504-448e-a953-3740111339fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:20:34.538175 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da49ff89-4504-448e-a953-3740111339fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:34.541801 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da49ff89-4504-448e-a953-3740111339fb admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:34.564228 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-b92ead9f-9d52-4c24-89c4-b730b2947901 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:34.565495 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-b92ead9f-9d52-4c24-89c4-b730b2947901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:34.571518 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 760/1522] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Oct 12 00:20:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 00:20:34.572605 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da49ff89-4504-448e-a953-3740111339fb admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:34.585363 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da49ff89-4504-448e-a953-3740111339fb admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:34.590301 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 763/1523] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:34.590777 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef25e5d-07a7-4999-84b2-b905912d8113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:34.592181 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef25e5d-07a7-4999-84b2-b905912d8113 admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:34.594575 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef25e5d-07a7-4999-84b2-b905912d8113 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:34.594575 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef25e5d-07a7-4999-84b2-b905912d8113 admin admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:34.594575 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef25e5d-07a7-4999-84b2-b905912d8113 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:34.595853 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef25e5d-07a7-4999-84b2-b905912d8113 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:34.596500 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92ead9f-9d52-4c24-89c4-b730b2947901 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:34.607669 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92ead9f-9d52-4c24-89c4-b730b2947901 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:34.619265 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d2a783-5b0c-4ce2-82f1-d93ed8ed1009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:34.619936 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d2a783-5b0c-4ce2-82f1-d93ed8ed1009 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:34.620217 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d2a783-5b0c-4ce2-82f1-d93ed8ed1009 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:34.621487 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d2a783-5b0c-4ce2-82f1-d93ed8ed1009 admin admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:34.621487 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d2a783-5b0c-4ce2-82f1-d93ed8ed1009 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:20:34.621779 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d2a783-5b0c-4ce2-82f1-d93ed8ed1009 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:34.624763 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d2a783-5b0c-4ce2-82f1-d93ed8ed1009 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:34.643330 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 764/1524] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Oct 12 00:20:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 00:20:34.648028 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d2a783-5b0c-4ce2-82f1-d93ed8ed1009 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:34.664458 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d2a783-5b0c-4ce2-82f1-d93ed8ed1009 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:34.682527 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be56054-39b4-4317-bf7c-cd9de9406dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:34.683867 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be56054-39b4-4317-bf7c-cd9de9406dd2 admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:34.684203 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be56054-39b4-4317-bf7c-cd9de9406dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:34.685691 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be56054-39b4-4317-bf7c-cd9de9406dd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:34.686806 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be56054-39b4-4317-bf7c-cd9de9406dd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:34.693113 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be56054-39b4-4317-bf7c-cd9de9406dd2 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:34.703186 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 761/1525] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:20:34 2022] POST /identity/v3/projects => generated 387 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:20:34.707593 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef25e5d-07a7-4999-84b2-b905912d8113 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:34.847707 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 765/1526] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:20:34 2022] POST /identity/v3/projects => generated 377 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:20:34.853121 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be56054-39b4-4317-bf7c-cd9de9406dd2 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:34.885488 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:34.915238 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef25e5d-07a7-4999-84b2-b905912d8113 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:35.007133 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be56054-39b4-4317-bf7c-cd9de9406dd2 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:35.014763 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f14028-c81f-4153-819f-9a88bdb019d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:35.017473 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f14028-c81f-4153-819f-9a88bdb019d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:35.018255 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f14028-c81f-4153-819f-9a88bdb019d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:35.018876 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f14028-c81f-4153-819f-9a88bdb019d5 admin admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:35.020026 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f14028-c81f-4153-819f-9a88bdb019d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:35.022925 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f14028-c81f-4153-819f-9a88bdb019d5 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:35.056912 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ddfa69-2488-420e-9969-3e79bd84201b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:35.057706 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ddfa69-2488-420e-9969-3e79bd84201b admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:35.057992 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ddfa69-2488-420e-9969-3e79bd84201b admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:35.058263 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ddfa69-2488-420e-9969-3e79bd84201b admin admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:35.058936 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ddfa69-2488-420e-9969-3e79bd84201b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:35.061829 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ddfa69-2488-420e-9969-3e79bd84201b admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:35.282880 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 762/1527] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:20:34 2022] POST /identity/v3/users => generated 294 bytes in 576 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 00:20:35.287124 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ddfa69-2488-420e-9969-3e79bd84201b admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:35.294261 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 766/1528] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:20:34 2022] POST /identity/v3/users => generated 289 bytes in 441 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:20:35.296595 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f14028-c81f-4153-819f-9a88bdb019d5 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:35.308034 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ddfa69-2488-420e-9969-3e79bd84201b admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:35.315627 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0717a08-a458-4312-858c-5992cc1c35f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:35.321300 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0717a08-a458-4312-858c-5992cc1c35f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:35.321774 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0717a08-a458-4312-858c-5992cc1c35f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:35.321892 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0717a08-a458-4312-858c-5992cc1c35f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:35.322565 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0717a08-a458-4312-858c-5992cc1c35f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:35.324389 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0717a08-a458-4312-858c-5992cc1c35f4 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:35.342813 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 763/1529] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:20:35 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:20:35.344564 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f14028-c81f-4153-819f-9a88bdb019d5 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:35.347744 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0717a08-a458-4312-858c-5992cc1c35f4 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:35.357479 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cd798e-3a8d-4b05-abf8-c78442015eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:35.358184 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cd798e-3a8d-4b05-abf8-c78442015eb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:35.358515 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cd798e-3a8d-4b05-abf8-c78442015eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:35.358786 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cd798e-3a8d-4b05-abf8-c78442015eb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:35.359695 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cd798e-3a8d-4b05-abf8-c78442015eb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:35.362881 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cd798e-3a8d-4b05-abf8-c78442015eb2 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:35.370857 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0717a08-a458-4312-858c-5992cc1c35f4 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:35.377015 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9113ca-58ee-4da8-8d78-1460dd3f0f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:35.377682 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9113ca-58ee-4da8-8d78-1460dd3f0f4d admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:35.377961 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9113ca-58ee-4da8-8d78-1460dd3f0f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:35.378229 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9113ca-58ee-4da8-8d78-1460dd3f0f4d admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:35.378954 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9113ca-58ee-4da8-8d78-1460dd3f0f4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:35.380896 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9113ca-58ee-4da8-8d78-1460dd3f0f4d admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:35.385560 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 767/1530] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:20:35 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:20:35.393339 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cd798e-3a8d-4b05-abf8-c78442015eb2 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:35.398707 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 764/1531] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:20:35 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:20:35.402709 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9113ca-58ee-4da8-8d78-1460dd3f0f4d admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:35.413326 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cd798e-3a8d-4b05-abf8-c78442015eb2 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:35.417011 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9113ca-58ee-4da8-8d78-1460dd3f0f4d admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:35.419088 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c578a577-87f9-42ab-b367-2ee7ff1aab6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:35.419914 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c578a577-87f9-42ab-b367-2ee7ff1aab6f admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:35.420348 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c578a577-87f9-42ab-b367-2ee7ff1aab6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:35.420769 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c578a577-87f9-42ab-b367-2ee7ff1aab6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:35.421571 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c578a577-87f9-42ab-b367-2ee7ff1aab6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:35.422604 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0f6049-6269-4f62-9815-fa82c3395ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:35.423552 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c578a577-87f9-42ab-b367-2ee7ff1aab6f admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:35.423816 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f6049-6269-4f62-9815-fa82c3395ebd admin admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:35.424205 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f6049-6269-4f62-9815-fa82c3395ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:35.424476 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f6049-6269-4f62-9815-fa82c3395ebd admin admin] PATH_INFO: `/v3/projects/4482691531e044679104fbaba77c30d3/users/4f637440d2a04fa2a5bf6eccf5e5a4a3/roles/ef689f96442945e8a7bc8cbfee13d909` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:35.438306 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 768/1532] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:20:35 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 00:20:35.444349 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c578a577-87f9-42ab-b367-2ee7ff1aab6f admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:35.451696 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0f6049-6269-4f62-9815-fa82c3395ebd admin admin] RBAC: Authorizing `identity:create_grant(project_id=4482691531e044679104fbaba77c30d3, user_id=4f637440d2a04fa2a5bf6eccf5e5a4a3, role_id=ef689f96442945e8a7bc8cbfee13d909)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:35.452573 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0f6049-6269-4f62-9815-fa82c3395ebd admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:35.461013 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c578a577-87f9-42ab-b367-2ee7ff1aab6f admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:35.465718 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421c17c7-4bc2-4f91-876a-d83edf4b840e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:35.466439 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c17c7-4bc2-4f91-876a-d83edf4b840e admin admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:35.467404 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c17c7-4bc2-4f91-876a-d83edf4b840e admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:35.467404 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c17c7-4bc2-4f91-876a-d83edf4b840e admin admin] PATH_INFO: `/v3/projects/5421643b82554363bc0fc8c142495be2/users/1ce1486fe2284fc1a5f76c702ad80444/roles/ef689f96442945e8a7bc8cbfee13d909` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:35.471376 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:35.482799 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421c17c7-4bc2-4f91-876a-d83edf4b840e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5421643b82554363bc0fc8c142495be2, user_id=1ce1486fe2284fc1a5f76c702ad80444, role_id=ef689f96442945e8a7bc8cbfee13d909)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:35.485481 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421c17c7-4bc2-4f91-876a-d83edf4b840e admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:35.572850 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 769/1533] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:20:35 2022] PUT /identity/v3/projects/5421643b82554363bc0fc8c142495be2/users/1ce1486fe2284fc1a5f76c702ad80444/roles/ef689f96442945e8a7bc8cbfee13d909 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 00:20:35.575110 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 765/1534] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:20:35 2022] PUT /identity/v3/projects/4482691531e044679104fbaba77c30d3/users/4f637440d2a04fa2a5bf6eccf5e5a4a3/roles/ef689f96442945e8a7bc8cbfee13d909 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 00:20:35.576832 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421c17c7-4bc2-4f91-876a-d83edf4b840e admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:35.582202 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-421c17c7-4bc2-4f91-876a-d83edf4b840e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:35.583671 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6236753-76cd-484b-afab-7f72f4ad312b None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:35.583958 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6236753-76cd-484b-afab-7f72f4ad312b None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:35.584232 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6236753-76cd-484b-afab-7f72f4ad312b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:35.618404 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-b6236753-76cd-484b-afab-7f72f4ad312b None None] MFA Rules not processed for user `1ce1486fe2284fc1a5f76c702ad80444`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:35.620413 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-b6236753-76cd-484b-afab-7f72f4ad312b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:35.661297 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421c17c7-4bc2-4f91-876a-d83edf4b840e admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:35.666317 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f968382-1a04-4838-b812-549c7ab3e679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:35.667022 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f968382-1a04-4838-b812-549c7ab3e679 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:35.667301 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f968382-1a04-4838-b812-549c7ab3e679 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:35.667606 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f968382-1a04-4838-b812-549c7ab3e679 admin admin] PATH_INFO: `/v3/projects/83468573643c456187416060e3b65170` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:35.668754 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f968382-1a04-4838-b812-549c7ab3e679 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83468573643c456187416060e3b65170)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:35.670718 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f968382-1a04-4838-b812-549c7ab3e679 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:35.685839 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 766/1535] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:35 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 00:20:35.687380 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c916cbe7-97ff-48fc-a52e-867421b53fe6 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:35.687668 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c916cbe7-97ff-48fc-a52e-867421b53fe6 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:35.687939 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c916cbe7-97ff-48fc-a52e-867421b53fe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:35.716252 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-c916cbe7-97ff-48fc-a52e-867421b53fe6 None None] MFA Rules not processed for user `4f637440d2a04fa2a5bf6eccf5e5a4a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:35.717910 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-c916cbe7-97ff-48fc-a52e-867421b53fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:35.784236 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 767/1536] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:35 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:35.785225 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c916cbe7-97ff-48fc-a52e-867421b53fe6 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:35.788075 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-4f968382-1a04-4838-b812-549c7ab3e679 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83468573643c456187416060e3b65170', 'request_id': 'req-4f968382-1a04-4838-b812-549c7ab3e679'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:35.795065 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-4f968382-1a04-4838-b812-549c7ab3e679 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83468573643c456187416060e3b65170', 'request_id': 'req-4f968382-1a04-4838-b812-549c7ab3e679'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:35.796594 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c916cbe7-97ff-48fc-a52e-867421b53fe6 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:35.852127 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390920f7-d4e4-48c3-a4e0-5ade8de56575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:35.853971 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390920f7-d4e4-48c3-a4e0-5ade8de56575 admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:35.853971 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390920f7-d4e4-48c3-a4e0-5ade8de56575 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:35.853971 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390920f7-d4e4-48c3-a4e0-5ade8de56575 admin admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:35.853971 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390920f7-d4e4-48c3-a4e0-5ade8de56575 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:35.854446 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-4f968382-1a04-4838-b812-549c7ab3e679 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83468573643c456187416060e3b65170', 'request_id': 'req-4f968382-1a04-4838-b812-549c7ab3e679'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:35.855818 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390920f7-d4e4-48c3-a4e0-5ade8de56575 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:35.869595 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-4f968382-1a04-4838-b812-549c7ab3e679 admin admin] The token cache is being invalidate because project 83468573643c456187416060e3b65170 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81747) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:20:35.869883 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-4f968382-1a04-4838-b812-549c7ab3e679 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:35.870555 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-4f968382-1a04-4838-b812-549c7ab3e679 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:35.873320 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 770/1537] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:20:35 2022] DELETE /identity/v3/projects/83468573643c456187416060e3b65170 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:35.880310 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f968382-1a04-4838-b812-549c7ab3e679 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:35.880310 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-4f968382-1a04-4838-b812-549c7ab3e679 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:35.893237 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f968382-1a04-4838-b812-549c7ab3e679 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:35.898419 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d13b81-b5ba-485e-aa3a-2ccdb2fc16ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:35.899055 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d13b81-b5ba-485e-aa3a-2ccdb2fc16ab admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:35.899330 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d13b81-b5ba-485e-aa3a-2ccdb2fc16ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:35.899627 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d13b81-b5ba-485e-aa3a-2ccdb2fc16ab admin admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:35.900250 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d13b81-b5ba-485e-aa3a-2ccdb2fc16ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:35.902211 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d13b81-b5ba-485e-aa3a-2ccdb2fc16ab admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:35.961427 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 768/1538] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:20:35 2022] POST /identity/v3/projects => generated 379 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:20:35.963227 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf38b52f-b297-4a99-96bb-528167d2cd54 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:35.963595 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf38b52f-b297-4a99-96bb-528167d2cd54 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:35.963890 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf38b52f-b297-4a99-96bb-528167d2cd54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:35.989314 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-bf38b52f-b297-4a99-96bb-528167d2cd54 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:35.990575 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 771/1539] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:20:35 2022] POST /identity/v3/projects => generated 385 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:20:35.990695 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-bf38b52f-b297-4a99-96bb-528167d2cd54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:35.991929 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d13b81-b5ba-485e-aa3a-2ccdb2fc16ab admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:35.996305 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-60d13b81-b5ba-485e-aa3a-2ccdb2fc16ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:36.072163 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 769/1540] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:36.073629 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf38b52f-b297-4a99-96bb-528167d2cd54 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:36.074862 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-bf38b52f-b297-4a99-96bb-528167d2cd54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:36.088516 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf38b52f-b297-4a99-96bb-528167d2cd54 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:36.089156 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d13b81-b5ba-485e-aa3a-2ccdb2fc16ab admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:36.094728 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472114e9-4779-43c4-98af-b0089d9b0ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:36.095119 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce55b859-620e-4e25-ba1f-06564241311d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:36.095948 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472114e9-4779-43c4-98af-b0089d9b0ab6 admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:36.096133 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472114e9-4779-43c4-98af-b0089d9b0ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:36.096351 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472114e9-4779-43c4-98af-b0089d9b0ab6 admin admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:36.096526 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce55b859-620e-4e25-ba1f-06564241311d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:36.096789 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce55b859-620e-4e25-ba1f-06564241311d admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:36.097049 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472114e9-4779-43c4-98af-b0089d9b0ab6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:36.097156 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce55b859-620e-4e25-ba1f-06564241311d admin admin] PATH_INFO: `/v3/users/6b79888e4a7042118ec0f66692a9b549` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:36.098225 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce55b859-620e-4e25-ba1f-06564241311d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b79888e4a7042118ec0f66692a9b549)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:36.098988 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472114e9-4779-43c4-98af-b0089d9b0ab6 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:36.100867 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce55b859-620e-4e25-ba1f-06564241311d admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:36.223452 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 770/1541] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:20:36 2022] POST /identity/v3/users => generated 296 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:20:36.224300 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472114e9-4779-43c4-98af-b0089d9b0ab6 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:36.234550 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472114e9-4779-43c4-98af-b0089d9b0ab6 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:36.239453 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895a7b2d-30aa-4e04-b782-029b1c4f07a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:36.240055 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a7b2d-30aa-4e04-b782-029b1c4f07a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:36.240495 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a7b2d-30aa-4e04-b782-029b1c4f07a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:36.243585 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a7b2d-30aa-4e04-b782-029b1c4f07a5 admin admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:36.243585 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895a7b2d-30aa-4e04-b782-029b1c4f07a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:36.244514 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895a7b2d-30aa-4e04-b782-029b1c4f07a5 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:36.324813 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 771/1542] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:20:36 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:20:36.326078 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f864905e-c129-4d86-9b5c-5cb0695c36cc None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:36.326353 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f864905e-c129-4d86-9b5c-5cb0695c36cc None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:36.326624 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f864905e-c129-4d86-9b5c-5cb0695c36cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:36.334096 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-ce55b859-620e-4e25-ba1f-06564241311d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b79888e4a7042118ec0f66692a9b549'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:36.335499 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-ce55b859-620e-4e25-ba1f-06564241311d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b79888e4a7042118ec0f66692a9b549'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:36.347514 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-f864905e-c129-4d86-9b5c-5cb0695c36cc None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:36.348270 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-ce55b859-620e-4e25-ba1f-06564241311d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b79888e4a7042118ec0f66692a9b549'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:36.348731 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-f864905e-c129-4d86-9b5c-5cb0695c36cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:36.357740 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 772/1543] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:36.358772 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f864905e-c129-4d86-9b5c-5cb0695c36cc None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:36.360013 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-f864905e-c129-4d86-9b5c-5cb0695c36cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:36.371447 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f864905e-c129-4d86-9b5c-5cb0695c36cc None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:36.376122 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4746689c-9c74-4021-bcd6-f2056446d189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:36.376718 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4746689c-9c74-4021-bcd6-f2056446d189 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:36.376999 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4746689c-9c74-4021-bcd6-f2056446d189 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:36.377273 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4746689c-9c74-4021-bcd6-f2056446d189 admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:36.377908 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4746689c-9c74-4021-bcd6-f2056446d189 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:36.379531 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4746689c-9c74-4021-bcd6-f2056446d189 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:36.391046 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 773/1544] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:20:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:20:36.391046 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4746689c-9c74-4021-bcd6-f2056446d189 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:36.393114 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-4746689c-9c74-4021-bcd6-f2056446d189 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:36.405368 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4746689c-9c74-4021-bcd6-f2056446d189 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:36.409932 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31015d7a-7f95-4f0b-a579-563c7f20a884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:36.410521 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31015d7a-7f95-4f0b-a579-563c7f20a884 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:36.410797 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31015d7a-7f95-4f0b-a579-563c7f20a884 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:36.411064 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31015d7a-7f95-4f0b-a579-563c7f20a884 admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:36.411467 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-ce55b859-620e-4e25-ba1f-06564241311d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b79888e4a7042118ec0f66692a9b549'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:36.411727 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31015d7a-7f95-4f0b-a579-563c7f20a884 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:36.412630 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-ce55b859-620e-4e25-ba1f-06564241311d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b79888e4a7042118ec0f66692a9b549'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:36.413307 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31015d7a-7f95-4f0b-a579-563c7f20a884 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:36.423820 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 774/1545] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:20:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:20:36.424619 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31015d7a-7f95-4f0b-a579-563c7f20a884 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:36.431689 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 772/1546] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:20:35 2022] DELETE /identity/v3/users/6b79888e4a7042118ec0f66692a9b549 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:36.432715 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce55b859-620e-4e25-ba1f-06564241311d admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:36.438654 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31015d7a-7f95-4f0b-a579-563c7f20a884 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:36.508122 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab104352-e61d-4acd-8a6d-5c350fd797aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:36.511418 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab104352-e61d-4acd-8a6d-5c350fd797aa admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:36.511418 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab104352-e61d-4acd-8a6d-5c350fd797aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:36.511418 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab104352-e61d-4acd-8a6d-5c350fd797aa admin admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:36.511418 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab104352-e61d-4acd-8a6d-5c350fd797aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:20:36.511418 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab104352-e61d-4acd-8a6d-5c350fd797aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:36.511811 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce55b859-620e-4e25-ba1f-06564241311d admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:36.512054 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab104352-e61d-4acd-8a6d-5c350fd797aa admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:36.516628 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ad1eca-6eb2-4bcf-ac5b-778584ec7c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:36.517369 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ad1eca-6eb2-4bcf-ac5b-778584ec7c41 admin admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:36.517732 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ad1eca-6eb2-4bcf-ac5b-778584ec7c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:36.518199 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ad1eca-6eb2-4bcf-ac5b-778584ec7c41 admin admin] PATH_INFO: `/v3/projects/44ec116e47ec46fb887e0bc11588a2a4/users/0974ca7c1f264a8885502fc150692a5a/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:36.532123 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 775/1547] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Oct 12 00:20:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:20:36.532123 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab104352-e61d-4acd-8a6d-5c350fd797aa admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:36.542327 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab104352-e61d-4acd-8a6d-5c350fd797aa admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:36.548147 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ad1eca-6eb2-4bcf-ac5b-778584ec7c41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44ec116e47ec46fb887e0bc11588a2a4, user_id=0974ca7c1f264a8885502fc150692a5a, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:36.551711 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ad1eca-6eb2-4bcf-ac5b-778584ec7c41 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:36.553044 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecb6d2b-7efe-4e87-953f-4092231e1b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:36.553917 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecb6d2b-7efe-4e87-953f-4092231e1b47 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:36.554590 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecb6d2b-7efe-4e87-953f-4092231e1b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:36.555160 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecb6d2b-7efe-4e87-953f-4092231e1b47 admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:36.555876 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecb6d2b-7efe-4e87-953f-4092231e1b47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:36.558402 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecb6d2b-7efe-4e87-953f-4092231e1b47 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:36.571073 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 776/1548] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:20:36 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:20:36.571073 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81097452-6db2-45e9-baac-a816b0e20f8a None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:36.571073 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81097452-6db2-45e9-baac-a816b0e20f8a None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:36.571073 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81097452-6db2-45e9-baac-a816b0e20f8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:36.594087 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-81097452-6db2-45e9-baac-a816b0e20f8a None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:36.595315 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-81097452-6db2-45e9-baac-a816b0e20f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:36.602925 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 777/1549] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:36.603736 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81097452-6db2-45e9-baac-a816b0e20f8a None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:36.612967 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81097452-6db2-45e9-baac-a816b0e20f8a None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:36.618133 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880c9897-b667-428e-93d4-9a53c170491c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:36.618736 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880c9897-b667-428e-93d4-9a53c170491c admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:36.619012 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880c9897-b667-428e-93d4-9a53c170491c admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:36.619276 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880c9897-b667-428e-93d4-9a53c170491c admin admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:36.619936 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880c9897-b667-428e-93d4-9a53c170491c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:36.621931 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880c9897-b667-428e-93d4-9a53c170491c admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:36.641848 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 773/1550] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:20:36 2022] PUT /identity/v3/projects/44ec116e47ec46fb887e0bc11588a2a4/users/0974ca7c1f264a8885502fc150692a5a/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:36.645666 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ad1eca-6eb2-4bcf-ac5b-778584ec7c41 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:36.721019 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ad1eca-6eb2-4bcf-ac5b-778584ec7c41 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:36.729952 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd41e2f2-ba66-4534-b88d-20f0b9d2f00f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:36.730558 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41e2f2-ba66-4534-b88d-20f0b9d2f00f admin admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:36.730831 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41e2f2-ba66-4534-b88d-20f0b9d2f00f admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:36.731097 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41e2f2-ba66-4534-b88d-20f0b9d2f00f admin admin] PATH_INFO: `/v3/projects/296f3fd4378b4da2a020039bd67613a3/users/15969b140423409b89e7491f0fb191e4/roles/ef689f96442945e8a7bc8cbfee13d909` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:36.733761 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 778/1551] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:20:36 2022] POST /identity/v3/projects => generated 383 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:20:36.734625 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880c9897-b667-428e-93d4-9a53c170491c admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:36.736558 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-880c9897-b667-428e-93d4-9a53c170491c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:36.751277 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd41e2f2-ba66-4534-b88d-20f0b9d2f00f admin admin] RBAC: Authorizing `identity:create_grant(project_id=296f3fd4378b4da2a020039bd67613a3, user_id=15969b140423409b89e7491f0fb191e4, role_id=ef689f96442945e8a7bc8cbfee13d909)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:36.753980 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd41e2f2-ba66-4534-b88d-20f0b9d2f00f admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:36.766086 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:36.800351 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880c9897-b667-428e-93d4-9a53c170491c admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:36.804418 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37b6769-e555-4f0c-919f-874bb3ddd49a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:36.805012 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37b6769-e555-4f0c-919f-874bb3ddd49a service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:36.805322 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37b6769-e555-4f0c-919f-874bb3ddd49a service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:36.805611 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37b6769-e555-4f0c-919f-874bb3ddd49a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:36.835012 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 774/1552] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:20:36 2022] PUT /identity/v3/projects/296f3fd4378b4da2a020039bd67613a3/users/15969b140423409b89e7491f0fb191e4/roles/ef689f96442945e8a7bc8cbfee13d909 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:36.835784 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd41e2f2-ba66-4534-b88d-20f0b9d2f00f admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:36.884997 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37b6769-e555-4f0c-919f-874bb3ddd49a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:36.886463 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37b6769-e555-4f0c-919f-874bb3ddd49a service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:36.898816 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 779/1553] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:20:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:20:36.899574 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37b6769-e555-4f0c-919f-874bb3ddd49a service neutron] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:36.905621 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd41e2f2-ba66-4534-b88d-20f0b9d2f00f admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:36.911117 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36019fd-74ad-4279-8b39-f963cbe72d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:36.911914 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36019fd-74ad-4279-8b39-f963cbe72d20 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:36.912338 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36019fd-74ad-4279-8b39-f963cbe72d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:36.912610 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36019fd-74ad-4279-8b39-f963cbe72d20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:36.913181 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37b6769-e555-4f0c-919f-874bb3ddd49a service neutron] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:36.913284 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36019fd-74ad-4279-8b39-f963cbe72d20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:36.914840 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36019fd-74ad-4279-8b39-f963cbe72d20 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:36.918126 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbebf89f-f86d-41aa-a696-0e0f2d6c7793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:36.919119 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbebf89f-f86d-41aa-a696-0e0f2d6c7793 admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:36.919434 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbebf89f-f86d-41aa-a696-0e0f2d6c7793 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:36.919718 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbebf89f-f86d-41aa-a696-0e0f2d6c7793 admin admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:36.920356 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbebf89f-f86d-41aa-a696-0e0f2d6c7793 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:36.922281 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbebf89f-f86d-41aa-a696-0e0f2d6c7793 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:36.930251 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 775/1554] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:20:36 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:20:36.930251 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce59ace-bbc0-487a-906f-cab9b6a18a7d None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:36.930251 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce59ace-bbc0-487a-906f-cab9b6a18a7d None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:36.930251 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce59ace-bbc0-487a-906f-cab9b6a18a7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:36.959882 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-3ce59ace-bbc0-487a-906f-cab9b6a18a7d None None] MFA Rules not processed for user `15969b140423409b89e7491f0fb191e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:36.961106 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-3ce59ace-bbc0-487a-906f-cab9b6a18a7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:37.002239 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 780/1555] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:20:36 2022] POST /identity/v3/users => generated 292 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:20:37.003142 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbebf89f-f86d-41aa-a696-0e0f2d6c7793 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:37.013981 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbebf89f-f86d-41aa-a696-0e0f2d6c7793 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:37.018846 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007c0b94-ef69-4d53-916d-ac415ccae66e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:37.019540 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007c0b94-ef69-4d53-916d-ac415ccae66e admin admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:37.019817 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007c0b94-ef69-4d53-916d-ac415ccae66e admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:37.020088 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007c0b94-ef69-4d53-916d-ac415ccae66e admin admin] PATH_INFO: `/v3/projects/44ec116e47ec46fb887e0bc11588a2a4/users/0974ca7c1f264a8885502fc150692a5a/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:37.022137 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007c0b94-ef69-4d53-916d-ac415ccae66e admin admin] RBAC: Authorizing `identity:create_grant(project_id=44ec116e47ec46fb887e0bc11588a2a4, user_id=0974ca7c1f264a8885502fc150692a5a, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:37.024832 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007c0b94-ef69-4d53-916d-ac415ccae66e admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:37.036163 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 781/1556] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:20:37 2022] PUT /identity/v3/projects/44ec116e47ec46fb887e0bc11588a2a4/users/0974ca7c1f264a8885502fc150692a5a/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:37.037134 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007c0b94-ef69-4d53-916d-ac415ccae66e admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:37.054184 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 776/1557] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:37.054184 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6910f1-42a8-43a9-9c1d-217a2d12472d None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:37.054184 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6910f1-42a8-43a9-9c1d-217a2d12472d None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:37.054184 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6910f1-42a8-43a9-9c1d-217a2d12472d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:37.071070 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:37.085973 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-2a6910f1-42a8-43a9-9c1d-217a2d12472d None None] MFA Rules not processed for user `0974ca7c1f264a8885502fc150692a5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:37.087210 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-2a6910f1-42a8-43a9-9c1d-217a2d12472d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:37.109214 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007c0b94-ef69-4d53-916d-ac415ccae66e admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:37.113838 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de4dcec-e491-4617-aa3c-a79afc417cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:37.114439 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de4dcec-e491-4617-aa3c-a79afc417cbe admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:37.114712 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de4dcec-e491-4617-aa3c-a79afc417cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:37.114979 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de4dcec-e491-4617-aa3c-a79afc417cbe admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:37.115648 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de4dcec-e491-4617-aa3c-a79afc417cbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:37.117451 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de4dcec-e491-4617-aa3c-a79afc417cbe admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:37.127689 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 782/1558] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:20:37 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:20:37.128561 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de4dcec-e491-4617-aa3c-a79afc417cbe admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:37.139125 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de4dcec-e491-4617-aa3c-a79afc417cbe admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:37.145013 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d974835-6ab5-411f-8b3c-576c0e1ecadd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:37.145639 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d974835-6ab5-411f-8b3c-576c0e1ecadd admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:37.145920 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d974835-6ab5-411f-8b3c-576c0e1ecadd admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:37.146189 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d974835-6ab5-411f-8b3c-576c0e1ecadd admin admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:37.146865 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d974835-6ab5-411f-8b3c-576c0e1ecadd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:37.148848 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d974835-6ab5-411f-8b3c-576c0e1ecadd admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:37.155517 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 777/1559] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:37 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:37.156535 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6910f1-42a8-43a9-9c1d-217a2d12472d None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:37.166726 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6910f1-42a8-43a9-9c1d-217a2d12472d None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:37.171582 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f80386e-8334-44bf-97f8-0fdf6e80bfe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:37.172191 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f80386e-8334-44bf-97f8-0fdf6e80bfe4 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:37.172469 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f80386e-8334-44bf-97f8-0fdf6e80bfe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:37.172736 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f80386e-8334-44bf-97f8-0fdf6e80bfe4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:37.173398 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f80386e-8334-44bf-97f8-0fdf6e80bfe4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:37.174984 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f80386e-8334-44bf-97f8-0fdf6e80bfe4 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:37.185575 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 778/1560] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:20:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:20:37.186490 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f80386e-8334-44bf-97f8-0fdf6e80bfe4 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:37.188192 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-3f80386e-8334-44bf-97f8-0fdf6e80bfe4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:37.242290 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:37.245729 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 783/1561] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:20:37 2022] POST /identity/v3/projects => generated 387 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:20:37.247880 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d974835-6ab5-411f-8b3c-576c0e1ecadd admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:37.250060 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f80386e-8334-44bf-97f8-0fdf6e80bfe4 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:37.320956 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7f69f9-199d-4ead-9b37-fa44ccf2fc81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:37.321683 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7f69f9-199d-4ead-9b37-fa44ccf2fc81 service cinder] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:37.321820 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7f69f9-199d-4ead-9b37-fa44ccf2fc81 service cinder] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:37.322294 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7f69f9-199d-4ead-9b37-fa44ccf2fc81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:37.340066 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d974835-6ab5-411f-8b3c-576c0e1ecadd admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:37.346606 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c511e3e-7fb8-4275-96db-984d6a75889a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:37.347292 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c511e3e-7fb8-4275-96db-984d6a75889a admin admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:37.347611 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c511e3e-7fb8-4275-96db-984d6a75889a admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:37.347891 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c511e3e-7fb8-4275-96db-984d6a75889a admin admin] PATH_INFO: `/v3/projects/dff76c0c26d34f41835465f2d900605f/users/4cf349d7b4bf48a580976dc4e0ab8d48/roles/ef689f96442945e8a7bc8cbfee13d909` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:37.366203 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c511e3e-7fb8-4275-96db-984d6a75889a admin admin] RBAC: Authorizing `identity:create_grant(project_id=dff76c0c26d34f41835465f2d900605f, user_id=4cf349d7b4bf48a580976dc4e0ab8d48, role_id=ef689f96442945e8a7bc8cbfee13d909)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:37.368946 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c511e3e-7fb8-4275-96db-984d6a75889a admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:37.399319 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7f69f9-199d-4ead-9b37-fa44ccf2fc81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:37.400746 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7f69f9-199d-4ead-9b37-fa44ccf2fc81 service cinder] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:37.417434 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 779/1562] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:20:37 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:20:37.418263 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7f69f9-199d-4ead-9b37-fa44ccf2fc81 service cinder] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:37.439408 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7f69f9-199d-4ead-9b37-fa44ccf2fc81 service cinder] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:37.443687 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 784/1563] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:20:37 2022] PUT /identity/v3/projects/dff76c0c26d34f41835465f2d900605f/users/4cf349d7b4bf48a580976dc4e0ab8d48/roles/ef689f96442945e8a7bc8cbfee13d909 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:37.444310 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c511e3e-7fb8-4275-96db-984d6a75889a admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:37.446247 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-7c511e3e-7fb8-4275-96db-984d6a75889a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:37.464352 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b0d6d8-0417-43d9-a674-9263385dc842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:37.464352 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b0d6d8-0417-43d9-a674-9263385dc842 admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:37.464352 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b0d6d8-0417-43d9-a674-9263385dc842 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:37.464352 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b0d6d8-0417-43d9-a674-9263385dc842 admin admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:37.465045 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b0d6d8-0417-43d9-a674-9263385dc842 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:37.465045 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b0d6d8-0417-43d9-a674-9263385dc842 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:37.510902 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c511e3e-7fb8-4275-96db-984d6a75889a admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:37.516118 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d2f637-735e-4dea-81f9-74e38858878d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:37.516746 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d2f637-735e-4dea-81f9-74e38858878d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:37.517020 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d2f637-735e-4dea-81f9-74e38858878d admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:37.517285 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d2f637-735e-4dea-81f9-74e38858878d admin admin] PATH_INFO: `/v3/projects/093c7ca07d6d4ae089e9c1f787d2e06c` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:37.518418 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d2f637-735e-4dea-81f9-74e38858878d admin admin] RBAC: Authorizing `identity:delete_project(project_id=093c7ca07d6d4ae089e9c1f787d2e06c)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:37.520343 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d2f637-735e-4dea-81f9-74e38858878d admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:37.564293 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 780/1564] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:20:37 2022] POST /identity/v3/users => generated 300 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:20:37.565609 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f63bad4-10e4-4ef1-a818-936410f7c8c0 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:37.565891 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f63bad4-10e4-4ef1-a818-936410f7c8c0 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:37.566267 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f63bad4-10e4-4ef1-a818-936410f7c8c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:37.604585 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-7f63bad4-10e4-4ef1-a818-936410f7c8c0 None None] MFA Rules not processed for user `4cf349d7b4bf48a580976dc4e0ab8d48`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:37.605878 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-7f63bad4-10e4-4ef1-a818-936410f7c8c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:37.688622 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 781/1565] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:37.689724 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f63bad4-10e4-4ef1-a818-936410f7c8c0 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:37.700888 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-60d2f637-735e-4dea-81f9-74e38858878d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '093c7ca07d6d4ae089e9c1f787d2e06c', 'request_id': 'req-60d2f637-735e-4dea-81f9-74e38858878d'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:37.715074 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-60d2f637-735e-4dea-81f9-74e38858878d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '093c7ca07d6d4ae089e9c1f787d2e06c', 'request_id': 'req-60d2f637-735e-4dea-81f9-74e38858878d'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:37.717594 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f63bad4-10e4-4ef1-a818-936410f7c8c0 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:37.816368 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4fdada-361e-4c95-b975-1c52d22f9c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:37.817019 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4fdada-361e-4c95-b975-1c52d22f9c74 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:37.817299 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4fdada-361e-4c95-b975-1c52d22f9c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:37.817568 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4fdada-361e-4c95-b975-1c52d22f9c74 admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:37.818209 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4fdada-361e-4c95-b975-1c52d22f9c74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:37.819897 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4fdada-361e-4c95-b975-1c52d22f9c74 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:37.828354 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-60d2f637-735e-4dea-81f9-74e38858878d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '093c7ca07d6d4ae089e9c1f787d2e06c', 'request_id': 'req-60d2f637-735e-4dea-81f9-74e38858878d'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:37.834391 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 782/1566] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:20:37 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:20:37.835387 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4fdada-361e-4c95-b975-1c52d22f9c74 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:37.843289 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-60d2f637-735e-4dea-81f9-74e38858878d admin admin] The token cache is being invalidate because project 093c7ca07d6d4ae089e9c1f787d2e06c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81746) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:20:37.844705 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-60d2f637-735e-4dea-81f9-74e38858878d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:37.844705 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-60d2f637-735e-4dea-81f9-74e38858878d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:37.847646 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 785/1567] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:20:37 2022] DELETE /identity/v3/projects/093c7ca07d6d4ae089e9c1f787d2e06c => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:37.848592 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d2f637-735e-4dea-81f9-74e38858878d admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:37.849986 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4fdada-361e-4c95-b975-1c52d22f9c74 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:37.850675 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-60d2f637-735e-4dea-81f9-74e38858878d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:37.855148 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da57122-4a39-4b68-ad85-59f4f7ff3d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:37.855807 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da57122-4a39-4b68-ad85-59f4f7ff3d1b admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:37.856179 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da57122-4a39-4b68-ad85-59f4f7ff3d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:37.857008 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da57122-4a39-4b68-ad85-59f4f7ff3d1b admin admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:37.858478 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da57122-4a39-4b68-ad85-59f4f7ff3d1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:37.860844 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da57122-4a39-4b68-ad85-59f4f7ff3d1b admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:37.866721 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d2f637-735e-4dea-81f9-74e38858878d admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:37.872984 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43ee2fa-181d-43f9-a1be-9eefc2755a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:37.873750 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43ee2fa-181d-43f9-a1be-9eefc2755a3b admin admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:37.874085 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43ee2fa-181d-43f9-a1be-9eefc2755a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:37.874429 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43ee2fa-181d-43f9-a1be-9eefc2755a3b admin admin] PATH_INFO: `/v3/projects/99208e8c337f4206a93a1ef96a3c13da/users/f5b81caad7c5408ab79d800ad57f0c4e/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:37.891984 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:37.896438 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43ee2fa-181d-43f9-a1be-9eefc2755a3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=99208e8c337f4206a93a1ef96a3c13da, user_id=f5b81caad7c5408ab79d800ad57f0c4e, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:37.899474 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43ee2fa-181d-43f9-a1be-9eefc2755a3b admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:37.978600 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 783/1568] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:20:37 2022] POST /identity/v3/projects => generated 385 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:20:37.978789 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 786/1569] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:20:37 2022] PUT /identity/v3/projects/99208e8c337f4206a93a1ef96a3c13da/users/f5b81caad7c5408ab79d800ad57f0c4e/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:37.979627 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da57122-4a39-4b68-ad85-59f4f7ff3d1b admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:37.983244 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43ee2fa-181d-43f9-a1be-9eefc2755a3b admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:37.983835 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-2da57122-4a39-4b68-ad85-59f4f7ff3d1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:37.985644 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-f43ee2fa-181d-43f9-a1be-9eefc2755a3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:38.054806 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da57122-4a39-4b68-ad85-59f4f7ff3d1b admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:38.055609 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43ee2fa-181d-43f9-a1be-9eefc2755a3b admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:38.059552 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4c6cb2-a612-4923-937a-19513e372cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:38.060196 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c6cb2-a612-4923-937a-19513e372cb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:38.060419 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3821c5-5a31-485f-862a-7b2b746943b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:38.060665 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c6cb2-a612-4923-937a-19513e372cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:38.060953 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c6cb2-a612-4923-937a-19513e372cb9 admin admin] PATH_INFO: `/v3/users/ac05a769b7fb41a3a0d1f756b0f54e0e` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:38.061064 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3821c5-5a31-485f-862a-7b2b746943b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:38.061287 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3821c5-5a31-485f-862a-7b2b746943b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:38.061555 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3821c5-5a31-485f-862a-7b2b746943b4 admin admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:38.063404 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4c6cb2-a612-4923-937a-19513e372cb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac05a769b7fb41a3a0d1f756b0f54e0e)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:38.063545 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3821c5-5a31-485f-862a-7b2b746943b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:38.064246 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3821c5-5a31-485f-862a-7b2b746943b4 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:38.064372 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4c6cb2-a612-4923-937a-19513e372cb9 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:38.165484 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 787/1570] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:20:37 2022] POST /identity/v3/users => generated 299 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:20:38.165484 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3821c5-5a31-485f-862a-7b2b746943b4 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:38.173223 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3821c5-5a31-485f-862a-7b2b746943b4 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:38.177778 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5952460b-e181-4ca1-aa57-9a5df48022d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:38.178350 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5952460b-e181-4ca1-aa57-9a5df48022d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:38.178629 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5952460b-e181-4ca1-aa57-9a5df48022d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:38.178902 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5952460b-e181-4ca1-aa57-9a5df48022d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:38.179557 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5952460b-e181-4ca1-aa57-9a5df48022d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:38.181150 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5952460b-e181-4ca1-aa57-9a5df48022d7 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:38.190973 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 788/1571] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:20:38 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:20:38.191805 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5952460b-e181-4ca1-aa57-9a5df48022d7 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:38.193427 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-5952460b-e181-4ca1-aa57-9a5df48022d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:38.206431 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5952460b-e181-4ca1-aa57-9a5df48022d7 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:38.211036 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d8dd87-a502-4615-967a-625dea9c36e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:38.211669 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d8dd87-a502-4615-967a-625dea9c36e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:38.217835 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d8dd87-a502-4615-967a-625dea9c36e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:38.217835 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d8dd87-a502-4615-967a-625dea9c36e6 admin admin] PATH_INFO: `/v3/users/1ce1486fe2284fc1a5f76c702ad80444` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:38.217835 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d8dd87-a502-4615-967a-625dea9c36e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ce1486fe2284fc1a5f76c702ad80444)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:38.217835 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d8dd87-a502-4615-967a-625dea9c36e6 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:38.297175 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-0d4c6cb2-a612-4923-937a-19513e372cb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac05a769b7fb41a3a0d1f756b0f54e0e'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:38.297831 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-0d4c6cb2-a612-4923-937a-19513e372cb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac05a769b7fb41a3a0d1f756b0f54e0e'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:38.309795 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-0d4c6cb2-a612-4923-937a-19513e372cb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac05a769b7fb41a3a0d1f756b0f54e0e'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:38.405433 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-0d4c6cb2-a612-4923-937a-19513e372cb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac05a769b7fb41a3a0d1f756b0f54e0e'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:38.406046 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-0d4c6cb2-a612-4923-937a-19513e372cb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac05a769b7fb41a3a0d1f756b0f54e0e'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:38.419452 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 784/1572] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:20:37 2022] DELETE /identity/v3/users/ac05a769b7fb41a3a0d1f756b0f54e0e => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:38.419756 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-25d8dd87-a502-4615-967a-625dea9c36e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ce1486fe2284fc1a5f76c702ad80444'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:38.420401 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4c6cb2-a612-4923-937a-19513e372cb9 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:38.421730 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-0d4c6cb2-a612-4923-937a-19513e372cb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:38.422875 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-25d8dd87-a502-4615-967a-625dea9c36e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ce1486fe2284fc1a5f76c702ad80444'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:38.428024 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:38.440243 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-25d8dd87-a502-4615-967a-625dea9c36e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ce1486fe2284fc1a5f76c702ad80444'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:38.498795 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4c6cb2-a612-4923-937a-19513e372cb9 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:38.503728 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc04abb-8375-4700-a5fe-6da89b3915d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:38.504346 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc04abb-8375-4700-a5fe-6da89b3915d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:38.504641 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc04abb-8375-4700-a5fe-6da89b3915d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:38.504916 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc04abb-8375-4700-a5fe-6da89b3915d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:38.505564 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc04abb-8375-4700-a5fe-6da89b3915d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:38.507236 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc04abb-8375-4700-a5fe-6da89b3915d3 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:38.518297 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 785/1573] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:20:38 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:20:38.522062 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc04abb-8375-4700-a5fe-6da89b3915d3 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:38.523930 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-3cc04abb-8375-4700-a5fe-6da89b3915d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:38.537268 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc04abb-8375-4700-a5fe-6da89b3915d3 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:38.541978 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-25d8dd87-a502-4615-967a-625dea9c36e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ce1486fe2284fc1a5f76c702ad80444'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:38.542625 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-25d8dd87-a502-4615-967a-625dea9c36e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ce1486fe2284fc1a5f76c702ad80444'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:38.548017 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc20ee89-76b0-4951-b194-ca44d950933e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:38.548963 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc20ee89-76b0-4951-b194-ca44d950933e admin admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:38.552112 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc20ee89-76b0-4951-b194-ca44d950933e admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:38.552112 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc20ee89-76b0-4951-b194-ca44d950933e admin admin] PATH_INFO: `/v3/projects/99208e8c337f4206a93a1ef96a3c13da/users/f5b81caad7c5408ab79d800ad57f0c4e/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:38.552804 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc20ee89-76b0-4951-b194-ca44d950933e admin admin] RBAC: Authorizing `identity:create_grant(project_id=99208e8c337f4206a93a1ef96a3c13da, user_id=f5b81caad7c5408ab79d800ad57f0c4e, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:38.555487 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc20ee89-76b0-4951-b194-ca44d950933e admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:38.559756 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 789/1574] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:20:38 2022] DELETE /identity/v3/users/1ce1486fe2284fc1a5f76c702ad80444 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:38.560651 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d8dd87-a502-4615-967a-625dea9c36e6 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:38.562030 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-25d8dd87-a502-4615-967a-625dea9c36e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:38.575408 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 786/1575] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:20:38 2022] PUT /identity/v3/projects/99208e8c337f4206a93a1ef96a3c13da/users/f5b81caad7c5408ab79d800ad57f0c4e/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:38.575408 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0beef4f-bba8-4ba4-9a2d-ecd647d5ab3e None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:38.575408 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0beef4f-bba8-4ba4-9a2d-ecd647d5ab3e None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:38.575408 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0beef4f-bba8-4ba4-9a2d-ecd647d5ab3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:38.589639 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-d0beef4f-bba8-4ba4-9a2d-ecd647d5ab3e None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:38.591440 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-d0beef4f-bba8-4ba4-9a2d-ecd647d5ab3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:38.645462 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d8dd87-a502-4615-967a-625dea9c36e6 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:38.676626 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 787/1576] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:38.676626 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9100bd-e6ff-427e-9c43-7c38afdac49a None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:38.676626 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9100bd-e6ff-427e-9c43-7c38afdac49a None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:38.676626 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9100bd-e6ff-427e-9c43-7c38afdac49a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:38.709879 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-9f9100bd-e6ff-427e-9c43-7c38afdac49a None None] MFA Rules not processed for user `f5b81caad7c5408ab79d800ad57f0c4e`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:38.711102 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-9f9100bd-e6ff-427e-9c43-7c38afdac49a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:38.728059 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394f8678-19b4-4fe2-84c2-6a9925d42792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:38.728870 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394f8678-19b4-4fe2-84c2-6a9925d42792 admin admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:38.729300 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394f8678-19b4-4fe2-84c2-6a9925d42792 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:38.729762 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394f8678-19b4-4fe2-84c2-6a9925d42792 admin admin] PATH_INFO: `/v3/projects/5e8a6efec6224399b60321ae8367a9d2/users/45453d4147d949e1a6cff860a0f86fa1/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:38.747970 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394f8678-19b4-4fe2-84c2-6a9925d42792 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e8a6efec6224399b60321ae8367a9d2, user_id=45453d4147d949e1a6cff860a0f86fa1, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:38.751027 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394f8678-19b4-4fe2-84c2-6a9925d42792 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:38.781754 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 788/1577] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:38 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:38.782735 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9100bd-e6ff-427e-9c43-7c38afdac49a None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:38.784273 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-9f9100bd-e6ff-427e-9c43-7c38afdac49a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:38.827711 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 790/1578] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:20:38 2022] PUT /identity/v3/projects/5e8a6efec6224399b60321ae8367a9d2/users/45453d4147d949e1a6cff860a0f86fa1/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:38.829100 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394f8678-19b4-4fe2-84c2-6a9925d42792 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:38.831943 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-394f8678-19b4-4fe2-84c2-6a9925d42792 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:38.843728 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9100bd-e6ff-427e-9c43-7c38afdac49a None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:38.902768 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394f8678-19b4-4fe2-84c2-6a9925d42792 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:38.908840 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0f964b-9aac-445b-a6a2-f8cdb8ca1760 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:38.908840 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0f964b-9aac-445b-a6a2-f8cdb8ca1760 service neutron] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:38.908840 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0f964b-9aac-445b-a6a2-f8cdb8ca1760 service neutron] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:38.908840 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0f964b-9aac-445b-a6a2-f8cdb8ca1760 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:38.909343 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e1ed65-7b8c-4384-b8e6-3032552df828 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:38.909343 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e1ed65-7b8c-4384-b8e6-3032552df828 service cinder] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:38.909343 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e1ed65-7b8c-4384-b8e6-3032552df828 service cinder] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:38.909343 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e1ed65-7b8c-4384-b8e6-3032552df828 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:38.980628 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0f964b-9aac-445b-a6a2-f8cdb8ca1760 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:38.982361 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0f964b-9aac-445b-a6a2-f8cdb8ca1760 service neutron] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:38.995041 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 789/1579] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:20:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:20:38.996229 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0f964b-9aac-445b-a6a2-f8cdb8ca1760 service neutron] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:39.006020 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0f964b-9aac-445b-a6a2-f8cdb8ca1760 service neutron] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:39.011196 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:39.017365 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e1ed65-7b8c-4384-b8e6-3032552df828 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:39.019691 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e1ed65-7b8c-4384-b8e6-3032552df828 service cinder] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:39.019865 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5e79ff-4f74-4bc3-8c9d-5a03471a6fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:39.020501 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e79ff-4f74-4bc3-8c9d-5a03471a6fa0 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:39.020778 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e79ff-4f74-4bc3-8c9d-5a03471a6fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:39.021049 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e79ff-4f74-4bc3-8c9d-5a03471a6fa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:39.021698 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5e79ff-4f74-4bc3-8c9d-5a03471a6fa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:39.023587 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5e79ff-4f74-4bc3-8c9d-5a03471a6fa0 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:39.031221 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 791/1580] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:20:38 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:20:39.032569 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e1ed65-7b8c-4384-b8e6-3032552df828 service cinder] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:39.034034 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-d9e1ed65-7b8c-4384-b8e6-3032552df828 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:39.043081 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 790/1581] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:20:38 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:20:39.044571 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5e79ff-4f74-4bc3-8c9d-5a03471a6fa0 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:39.050157 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e1ed65-7b8c-4384-b8e6-3032552df828 service cinder] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:39.056632 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbb8306-5915-4080-8533-3644543255be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:39.057782 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbb8306-5915-4080-8533-3644543255be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:39.058809 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5e79ff-4f74-4bc3-8c9d-5a03471a6fa0 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:39.059249 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbb8306-5915-4080-8533-3644543255be admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:39.060196 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbb8306-5915-4080-8533-3644543255be admin admin] PATH_INFO: `/v3/projects/ae7c15f10b244e03a6a2fb3cfc73b24d` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:39.061682 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbb8306-5915-4080-8533-3644543255be admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae7c15f10b244e03a6a2fb3cfc73b24d)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:39.064583 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbb8306-5915-4080-8533-3644543255be admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:39.070608 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eef237f-8052-4c62-bea1-0fc6de2a9456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:39.070608 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eef237f-8052-4c62-bea1-0fc6de2a9456 admin admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:39.070608 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eef237f-8052-4c62-bea1-0fc6de2a9456 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:39.070608 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eef237f-8052-4c62-bea1-0fc6de2a9456 admin admin] PATH_INFO: `/v3/projects/5e8a6efec6224399b60321ae8367a9d2/users/45453d4147d949e1a6cff860a0f86fa1/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:39.070971 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eef237f-8052-4c62-bea1-0fc6de2a9456 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e8a6efec6224399b60321ae8367a9d2, user_id=45453d4147d949e1a6cff860a0f86fa1, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:39.073262 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eef237f-8052-4c62-bea1-0fc6de2a9456 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:39.087471 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 791/1582] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:20:39 2022] PUT /identity/v3/projects/5e8a6efec6224399b60321ae8367a9d2/users/45453d4147d949e1a6cff860a0f86fa1/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:39.100135 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b523cb69-e660-4987-8e21-8a4195cfc236 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:39.100534 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b523cb69-e660-4987-8e21-8a4195cfc236 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:39.100840 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b523cb69-e660-4987-8e21-8a4195cfc236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:39.136343 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-b523cb69-e660-4987-8e21-8a4195cfc236 None None] MFA Rules not processed for user `45453d4147d949e1a6cff860a0f86fa1`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:39.139076 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-b523cb69-e660-4987-8e21-8a4195cfc236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:39.220299 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-dfbb8306-5915-4080-8533-3644543255be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae7c15f10b244e03a6a2fb3cfc73b24d', 'request_id': 'req-dfbb8306-5915-4080-8533-3644543255be'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:39.223434 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 792/1583] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:39 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:39.232082 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b523cb69-e660-4987-8e21-8a4195cfc236 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:39.233633 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-dfbb8306-5915-4080-8533-3644543255be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae7c15f10b244e03a6a2fb3cfc73b24d', 'request_id': 'req-dfbb8306-5915-4080-8533-3644543255be'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:39.233882 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-b523cb69-e660-4987-8e21-8a4195cfc236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:39.294561 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b523cb69-e660-4987-8e21-8a4195cfc236 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:39.302135 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-dfbb8306-5915-4080-8533-3644543255be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae7c15f10b244e03a6a2fb3cfc73b24d', 'request_id': 'req-dfbb8306-5915-4080-8533-3644543255be'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:39.320045 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-dfbb8306-5915-4080-8533-3644543255be admin admin] The token cache is being invalidate because project ae7c15f10b244e03a6a2fb3cfc73b24d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81746) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:20:39.320342 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-dfbb8306-5915-4080-8533-3644543255be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:39.321030 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-dfbb8306-5915-4080-8533-3644543255be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:39.322751 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 792/1584] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:20:39 2022] DELETE /identity/v3/projects/ae7c15f10b244e03a6a2fb3cfc73b24d => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:39.329558 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4466b3-f46d-45ae-833a-7d0c93aef758 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:39.329964 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4466b3-f46d-45ae-833a-7d0c93aef758 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:39.330339 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4466b3-f46d-45ae-833a-7d0c93aef758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:39.352458 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-af4466b3-f46d-45ae-833a-7d0c93aef758 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:39.354081 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-af4466b3-f46d-45ae-833a-7d0c93aef758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:39.359535 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e233d341-30e7-4140-8ac3-9684f3984d70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:39.360167 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e233d341-30e7-4140-8ac3-9684f3984d70 service cinder] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:39.360446 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e233d341-30e7-4140-8ac3-9684f3984d70 service cinder] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:39.360716 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e233d341-30e7-4140-8ac3-9684f3984d70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:39.362768 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-e233d341-30e7-4140-8ac3-9684f3984d70 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:39.418538 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:39.451693 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e233d341-30e7-4140-8ac3-9684f3984d70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:39.453634 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e233d341-30e7-4140-8ac3-9684f3984d70 service cinder] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:39.478803 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 793/1586] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:20:39 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 00:20:39.478935 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 793/1586] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:39.488250 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95f25e2-61a4-4aaf-9b08-33741f254444 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:39.490345 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95f25e2-61a4-4aaf-9b08-33741f254444 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:39.490345 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95f25e2-61a4-4aaf-9b08-33741f254444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:39.525157 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-b95f25e2-61a4-4aaf-9b08-33741f254444 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:39.527140 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-b95f25e2-61a4-4aaf-9b08-33741f254444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:39.545079 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 794/1587] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:39.548907 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4466b3-f46d-45ae-833a-7d0c93aef758 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:39.562005 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4466b3-f46d-45ae-833a-7d0c93aef758 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:39.570958 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6973c06-5cd8-46d7-8370-dca0b9492a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:39.570958 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6973c06-5cd8-46d7-8370-dca0b9492a52 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:39.570958 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6973c06-5cd8-46d7-8370-dca0b9492a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:39.570958 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6973c06-5cd8-46d7-8370-dca0b9492a52 admin admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:39.571790 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6973c06-5cd8-46d7-8370-dca0b9492a52 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:20:39.571790 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6973c06-5cd8-46d7-8370-dca0b9492a52 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:39.573759 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6973c06-5cd8-46d7-8370-dca0b9492a52 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:39.591644 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:39.591867 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 794/1588] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Oct 12 00:20:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:20:39.592946 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95f25e2-61a4-4aaf-9b08-33741f254444 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:39.604888 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95f25e2-61a4-4aaf-9b08-33741f254444 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:39.610650 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab98e21-ce6d-410b-93e4-3eaa827d20ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:39.611117 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab98e21-ce6d-410b-93e4-3eaa827d20ab admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:39.611508 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab98e21-ce6d-410b-93e4-3eaa827d20ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:39.611723 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab98e21-ce6d-410b-93e4-3eaa827d20ab admin admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:39.612363 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab98e21-ce6d-410b-93e4-3eaa827d20ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:39.614309 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab98e21-ce6d-410b-93e4-3eaa827d20ab admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:39.728401 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6973c06-5cd8-46d7-8370-dca0b9492a52 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:39.730010 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 795/1589] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:20:39 2022] POST /identity/v3/projects => generated 381 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:20:39.733162 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-f6973c06-5cd8-46d7-8370-dca0b9492a52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:39.742533 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab98e21-ce6d-410b-93e4-3eaa827d20ab admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:39.756762 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6973c06-5cd8-46d7-8370-dca0b9492a52 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:39.838434 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab98e21-ce6d-410b-93e4-3eaa827d20ab admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:39.846265 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff9c046-2349-493d-b345-3237920541c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:39.846779 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b740136a-cc8c-40de-b1ff-91a0318d186e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:39.847253 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff9c046-2349-493d-b345-3237920541c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:39.847253 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff9c046-2349-493d-b345-3237920541c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:39.847446 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff9c046-2349-493d-b345-3237920541c3 admin admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:39.847528 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b740136a-cc8c-40de-b1ff-91a0318d186e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:39.847740 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b740136a-cc8c-40de-b1ff-91a0318d186e admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:39.848010 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b740136a-cc8c-40de-b1ff-91a0318d186e admin admin] PATH_INFO: `/v3/projects/5421643b82554363bc0fc8c142495be2` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:39.848112 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff9c046-2349-493d-b345-3237920541c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:39.849184 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b740136a-cc8c-40de-b1ff-91a0318d186e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5421643b82554363bc0fc8c142495be2)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:39.851479 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b740136a-cc8c-40de-b1ff-91a0318d186e admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:39.855644 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff9c046-2349-493d-b345-3237920541c3 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:39.946583 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 796/1590] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:20:39 2022] POST /identity/v3/users => generated 291 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:20:39.950399 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff9c046-2349-493d-b345-3237920541c3 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:39.959512 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-b740136a-cc8c-40de-b1ff-91a0318d186e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5421643b82554363bc0fc8c142495be2', 'request_id': 'req-b740136a-cc8c-40de-b1ff-91a0318d186e'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:39.960163 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff9c046-2349-493d-b345-3237920541c3 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:39.966471 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-b740136a-cc8c-40de-b1ff-91a0318d186e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5421643b82554363bc0fc8c142495be2', 'request_id': 'req-b740136a-cc8c-40de-b1ff-91a0318d186e'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:40.016661 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79b880f-534e-46a6-b70b-dd16155a7db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:40.017193 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79b880f-534e-46a6-b70b-dd16155a7db5 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:40.017472 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79b880f-534e-46a6-b70b-dd16155a7db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:40.017821 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79b880f-534e-46a6-b70b-dd16155a7db5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:40.018371 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79b880f-534e-46a6-b70b-dd16155a7db5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:40.020002 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79b880f-534e-46a6-b70b-dd16155a7db5 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:40.034197 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 797/1591] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:20:39 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 00:20:40.035187 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79b880f-534e-46a6-b70b-dd16155a7db5 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:40.045437 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79b880f-534e-46a6-b70b-dd16155a7db5 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:40.047897 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-b740136a-cc8c-40de-b1ff-91a0318d186e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5421643b82554363bc0fc8c142495be2', 'request_id': 'req-b740136a-cc8c-40de-b1ff-91a0318d186e'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:40.051096 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f0e93b-4919-439d-8c8b-b569368b12be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:40.051907 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f0e93b-4919-439d-8c8b-b569368b12be admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:40.052407 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f0e93b-4919-439d-8c8b-b569368b12be admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:40.052966 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f0e93b-4919-439d-8c8b-b569368b12be admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:40.053739 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f0e93b-4919-439d-8c8b-b569368b12be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:40.056317 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f0e93b-4919-439d-8c8b-b569368b12be admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:40.067418 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 798/1592] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:20:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:20:40.076931 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-b740136a-cc8c-40de-b1ff-91a0318d186e admin admin] The token cache is being invalidate because project 5421643b82554363bc0fc8c142495be2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81746) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:20:40.077422 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-b740136a-cc8c-40de-b1ff-91a0318d186e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:40.078269 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-b740136a-cc8c-40de-b1ff-91a0318d186e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:40.079150 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f0e93b-4919-439d-8c8b-b569368b12be admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:40.083701 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-97f0e93b-4919-439d-8c8b-b569368b12be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:40.085626 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 795/1593] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:20:39 2022] DELETE /identity/v3/projects/5421643b82554363bc0fc8c142495be2 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:40.087213 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b740136a-cc8c-40de-b1ff-91a0318d186e admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:40.088760 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-b740136a-cc8c-40de-b1ff-91a0318d186e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:40.101530 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f0e93b-4919-439d-8c8b-b569368b12be admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:40.103987 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b740136a-cc8c-40de-b1ff-91a0318d186e admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:40.109947 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce932fc9-be3d-4512-a433-4fedf9516285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:40.110517 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4268658a-99a3-4970-aff5-7323a6f0431d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:40.110643 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce932fc9-be3d-4512-a433-4fedf9516285 admin admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:40.110932 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce932fc9-be3d-4512-a433-4fedf9516285 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:40.111680 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4268658a-99a3-4970-aff5-7323a6f0431d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:40.111856 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce932fc9-be3d-4512-a433-4fedf9516285 admin admin] PATH_INFO: `/v3/projects/035b882afc6941fa94810b16ca6fef2c/users/349651cc16d74830a983e7e4cd05c199/roles/ef689f96442945e8a7bc8cbfee13d909` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:40.112260 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4268658a-99a3-4970-aff5-7323a6f0431d admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:40.112675 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4268658a-99a3-4970-aff5-7323a6f0431d admin admin] PATH_INFO: `/v3/users/0974ca7c1f264a8885502fc150692a5a` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:40.114003 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4268658a-99a3-4970-aff5-7323a6f0431d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0974ca7c1f264a8885502fc150692a5a)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:40.117684 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4268658a-99a3-4970-aff5-7323a6f0431d admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:40.157386 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce932fc9-be3d-4512-a433-4fedf9516285 admin admin] RBAC: Authorizing `identity:create_grant(project_id=035b882afc6941fa94810b16ca6fef2c, user_id=349651cc16d74830a983e7e4cd05c199, role_id=ef689f96442945e8a7bc8cbfee13d909)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:40.182005 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce932fc9-be3d-4512-a433-4fedf9516285 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:41.080071 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 799/1594] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:20:40 2022] PUT /identity/v3/projects/035b882afc6941fa94810b16ca6fef2c/users/349651cc16d74830a983e7e4cd05c199/roles/ef689f96442945e8a7bc8cbfee13d909 => generated 0 bytes in 1002 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:41.080888 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce932fc9-be3d-4512-a433-4fedf9516285 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:41.082509 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-ce932fc9-be3d-4512-a433-4fedf9516285 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:41.142581 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce932fc9-be3d-4512-a433-4fedf9516285 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:41.146201 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410655ef-4204-49f1-b69b-bd8e07b992b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:41.146800 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410655ef-4204-49f1-b69b-bd8e07b992b0 service cinder] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:41.147076 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410655ef-4204-49f1-b69b-bd8e07b992b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:41.147344 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410655ef-4204-49f1-b69b-bd8e07b992b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:41.149097 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-410655ef-4204-49f1-b69b-bd8e07b992b0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:41.174007 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-4268658a-99a3-4970-aff5-7323a6f0431d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0974ca7c1f264a8885502fc150692a5a'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:41.175253 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-4268658a-99a3-4970-aff5-7323a6f0431d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0974ca7c1f264a8885502fc150692a5a'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:41.193889 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-4268658a-99a3-4970-aff5-7323a6f0431d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0974ca7c1f264a8885502fc150692a5a'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:41.225481 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410655ef-4204-49f1-b69b-bd8e07b992b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:41.227723 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410655ef-4204-49f1-b69b-bd8e07b992b0 service cinder] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:41.237431 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 800/1595] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:20:41 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:20:41.239003 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdca45e-4cb5-41af-8454-8bfd66f68198 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:41.239278 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdca45e-4cb5-41af-8454-8bfd66f68198 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:41.239707 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdca45e-4cb5-41af-8454-8bfd66f68198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:41.270770 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-afdca45e-4cb5-41af-8454-8bfd66f68198 None None] MFA Rules not processed for user `349651cc16d74830a983e7e4cd05c199`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:41.272103 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-afdca45e-4cb5-41af-8454-8bfd66f68198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:41.302487 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-4268658a-99a3-4970-aff5-7323a6f0431d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0974ca7c1f264a8885502fc150692a5a'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:41.303224 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-4268658a-99a3-4970-aff5-7323a6f0431d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0974ca7c1f264a8885502fc150692a5a'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:41.329394 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 796/1596] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:20:40 2022] DELETE /identity/v3/users/0974ca7c1f264a8885502fc150692a5a => generated 0 bytes in 1240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:41.384016 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 801/1597] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:41 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:41.388759 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4268658a-99a3-4970-aff5-7323a6f0431d admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:41.390930 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-4268658a-99a3-4970-aff5-7323a6f0431d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:41.486454 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4268658a-99a3-4970-aff5-7323a6f0431d admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:41.492537 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77e329e-c1c0-4941-98ce-4ac1b00306ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:41.493792 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e329e-c1c0-4941-98ce-4ac1b00306ad admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:41.494313 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e329e-c1c0-4941-98ce-4ac1b00306ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:41.494958 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e329e-c1c0-4941-98ce-4ac1b00306ad admin admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:41.495904 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77e329e-c1c0-4941-98ce-4ac1b00306ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:41.498314 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77e329e-c1c0-4941-98ce-4ac1b00306ad admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:41.622166 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 797/1598] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:20:41 2022] POST /identity/v3/projects => generated 379 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:20:41.626036 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdca45e-4cb5-41af-8454-8bfd66f68198 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:41.722147 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdca45e-4cb5-41af-8454-8bfd66f68198 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:41.820075 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaca80a-f159-4ad6-97fb-de2a3f79bc56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:41.820875 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaca80a-f159-4ad6-97fb-de2a3f79bc56 admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:41.821594 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaca80a-f159-4ad6-97fb-de2a3f79bc56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:41.821594 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaca80a-f159-4ad6-97fb-de2a3f79bc56 admin admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:41.822265 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaca80a-f159-4ad6-97fb-de2a3f79bc56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:41.824152 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaca80a-f159-4ad6-97fb-de2a3f79bc56 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:41.865498 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:41.944204 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 802/1599] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:20:41 2022] POST /identity/v3/users => generated 296 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:20:41.945153 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77e329e-c1c0-4941-98ce-4ac1b00306ad admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:41.971957 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77e329e-c1c0-4941-98ce-4ac1b00306ad admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:41.976687 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2591b153-a2b0-4a20-9eae-8f169cab560b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:41.977351 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2591b153-a2b0-4a20-9eae-8f169cab560b admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:41.977644 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2591b153-a2b0-4a20-9eae-8f169cab560b admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:41.977917 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2591b153-a2b0-4a20-9eae-8f169cab560b admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:41.978596 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2591b153-a2b0-4a20-9eae-8f169cab560b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:41.980313 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2591b153-a2b0-4a20-9eae-8f169cab560b admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:41.994844 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 798/1600] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:20:41 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:20:41.995658 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaca80a-f159-4ad6-97fb-de2a3f79bc56 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:42.002057 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2591b153-a2b0-4a20-9eae-8f169cab560b admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:42.006211 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-2591b153-a2b0-4a20-9eae-8f169cab560b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:42.009875 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaca80a-f159-4ad6-97fb-de2a3f79bc56 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:42.015645 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8031a0e9-8abc-49d1-9f9a-9b04a422f52f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:42.017437 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8031a0e9-8abc-49d1-9f9a-9b04a422f52f admin admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:42.017437 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8031a0e9-8abc-49d1-9f9a-9b04a422f52f admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:42.017437 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8031a0e9-8abc-49d1-9f9a-9b04a422f52f admin admin] PATH_INFO: `/v3/projects/94e0e8b6283f47de8d979032c8098d30/users/9b744fdcd677444e89933e7193c4b230/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:42.022887 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2591b153-a2b0-4a20-9eae-8f169cab560b admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:42.028557 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071332f0-8378-4c24-9624-e2c7b1e2c45a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:42.029421 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071332f0-8378-4c24-9624-e2c7b1e2c45a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:42.029698 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071332f0-8378-4c24-9624-e2c7b1e2c45a admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:42.029965 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071332f0-8378-4c24-9624-e2c7b1e2c45a admin admin] PATH_INFO: `/v3/projects/44ec116e47ec46fb887e0bc11588a2a4` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:42.031311 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071332f0-8378-4c24-9624-e2c7b1e2c45a admin admin] RBAC: Authorizing `identity:delete_project(project_id=44ec116e47ec46fb887e0bc11588a2a4)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:42.033865 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071332f0-8378-4c24-9624-e2c7b1e2c45a admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:42.035263 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8031a0e9-8abc-49d1-9f9a-9b04a422f52f admin admin] RBAC: Authorizing `identity:create_grant(project_id=94e0e8b6283f47de8d979032c8098d30, user_id=9b744fdcd677444e89933e7193c4b230, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:42.038423 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8031a0e9-8abc-49d1-9f9a-9b04a422f52f admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:42.139800 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 803/1601] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:20:41 2022] PUT /identity/v3/projects/94e0e8b6283f47de8d979032c8098d30/users/9b744fdcd677444e89933e7193c4b230/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:42.141091 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e7c432-eb13-44f0-adc8-d81ddf41cbc2 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:42.141331 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e7c432-eb13-44f0-adc8-d81ddf41cbc2 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:42.141617 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e7c432-eb13-44f0-adc8-d81ddf41cbc2 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:42.142941 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 804/1602] 10.222.0.36 () {54 vars in 921 bytes} [Wed Oct 12 00:20:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:20:42.143692 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e7c432-eb13-44f0-adc8-d81ddf41cbc2 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:42.172807 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-071332f0-8378-4c24-9624-e2c7b1e2c45a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44ec116e47ec46fb887e0bc11588a2a4', 'request_id': 'req-071332f0-8378-4c24-9624-e2c7b1e2c45a'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:42.187098 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-071332f0-8378-4c24-9624-e2c7b1e2c45a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44ec116e47ec46fb887e0bc11588a2a4', 'request_id': 'req-071332f0-8378-4c24-9624-e2c7b1e2c45a'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:42.240068 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:42.245445 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e7c432-eb13-44f0-adc8-d81ddf41cbc2 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:42.268180 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-071332f0-8378-4c24-9624-e2c7b1e2c45a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44ec116e47ec46fb887e0bc11588a2a4', 'request_id': 'req-071332f0-8378-4c24-9624-e2c7b1e2c45a'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:42.282225 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-071332f0-8378-4c24-9624-e2c7b1e2c45a admin admin] The token cache is being invalidate because project 44ec116e47ec46fb887e0bc11588a2a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81746) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:20:42.282577 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-071332f0-8378-4c24-9624-e2c7b1e2c45a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:42.283538 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-071332f0-8378-4c24-9624-e2c7b1e2c45a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:42.286527 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 799/1603] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:20:41 2022] DELETE /identity/v3/projects/44ec116e47ec46fb887e0bc11588a2a4 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:42.287909 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af0810-35d0-4d9a-8611-568885615ba6 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:42.288198 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af0810-35d0-4d9a-8611-568885615ba6 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:42.288475 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af0810-35d0-4d9a-8611-568885615ba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:42.318676 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-71af0810-35d0-4d9a-8611-568885615ba6 None None] MFA Rules not processed for user `a515b75aa3944912ad2364ffd7924f40`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:42.320236 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-71af0810-35d0-4d9a-8611-568885615ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:42.331125 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ce1044-0061-4e04-8f9b-955583e2149e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:42.331125 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ce1044-0061-4e04-8f9b-955583e2149e admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:42.331125 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ce1044-0061-4e04-8f9b-955583e2149e admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:42.331125 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ce1044-0061-4e04-8f9b-955583e2149e admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:42.331512 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ce1044-0061-4e04-8f9b-955583e2149e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:42.331512 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ce1044-0061-4e04-8f9b-955583e2149e admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:42.338132 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 805/1604] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:20:42 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:20:42.340053 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb359c59-236d-45e7-84fb-c55db9ef83aa None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:42.340053 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb359c59-236d-45e7-84fb-c55db9ef83aa None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:42.340053 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb359c59-236d-45e7-84fb-c55db9ef83aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:42.366368 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-bb359c59-236d-45e7-84fb-c55db9ef83aa None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:42.366368 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-bb359c59-236d-45e7-84fb-c55db9ef83aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:42.371254 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 806/1605] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:42.378457 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb359c59-236d-45e7-84fb-c55db9ef83aa None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:42.378457 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-bb359c59-236d-45e7-84fb-c55db9ef83aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:42.388076 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb359c59-236d-45e7-84fb-c55db9ef83aa None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:42.394121 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ba6106-48aa-41f3-b9e8-6f42402e8693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:42.394857 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ba6106-48aa-41f3-b9e8-6f42402e8693 admin admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:42.395200 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ba6106-48aa-41f3-b9e8-6f42402e8693 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:42.395586 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ba6106-48aa-41f3-b9e8-6f42402e8693 admin admin] PATH_INFO: `/v3/projects/94e0e8b6283f47de8d979032c8098d30/users/9b744fdcd677444e89933e7193c4b230/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:42.397776 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ba6106-48aa-41f3-b9e8-6f42402e8693 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94e0e8b6283f47de8d979032c8098d30, user_id=9b744fdcd677444e89933e7193c4b230, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:42.400868 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ba6106-48aa-41f3-b9e8-6f42402e8693 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:42.406274 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 800/1606] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Oct 12 00:20:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:42.407574 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed6a20a-8911-462b-93fa-20006f0a8fbc None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:42.407858 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed6a20a-8911-462b-93fa-20006f0a8fbc None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:42.408129 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed6a20a-8911-462b-93fa-20006f0a8fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:42.414323 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 807/1607] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:20:42 2022] PUT /identity/v3/projects/94e0e8b6283f47de8d979032c8098d30/users/9b744fdcd677444e89933e7193c4b230/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:42.420946 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ba6106-48aa-41f3-b9e8-6f42402e8693 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:42.424290 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-96ba6106-48aa-41f3-b9e8-6f42402e8693 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:42.437171 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-fed6a20a-8911-462b-93fa-20006f0a8fbc None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:42.438400 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-fed6a20a-8911-462b-93fa-20006f0a8fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:42.516383 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 801/1608] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:42.517489 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f25e0eb-407f-4334-87c4-617c46affa4c None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:42.517823 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f25e0eb-407f-4334-87c4-617c46affa4c None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:42.518189 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f25e0eb-407f-4334-87c4-617c46affa4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:42.521378 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ba6106-48aa-41f3-b9e8-6f42402e8693 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:42.527230 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc163eb6-05f9-4615-b6a8-2a85dc636c50 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a515b75aa3944912ad2364ffd7924f40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:42.527995 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc163eb6-05f9-4615-b6a8-2a85dc636c50 service nova] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:42.528262 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc163eb6-05f9-4615-b6a8-2a85dc636c50 service nova] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:42.528547 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc163eb6-05f9-4615-b6a8-2a85dc636c50 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:42.550722 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc163eb6-05f9-4615-b6a8-2a85dc636c50 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:42.552506 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc163eb6-05f9-4615-b6a8-2a85dc636c50 service nova] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:42.555520 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-5f25e0eb-407f-4334-87c4-617c46affa4c None None] MFA Rules not processed for user `9b744fdcd677444e89933e7193c4b230`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:42.556756 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-5f25e0eb-407f-4334-87c4-617c46affa4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:42.566453 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 808/1609] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:20:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 00:20:42.569588 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc163eb6-05f9-4615-b6a8-2a85dc636c50 service nova] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:42.579802 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc163eb6-05f9-4615-b6a8-2a85dc636c50 service nova] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:42.586170 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75388e5-e6f3-440a-a9e0-834668d94887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:42.587864 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75388e5-e6f3-440a-a9e0-834668d94887 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:42.591749 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75388e5-e6f3-440a-a9e0-834668d94887 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:42.592482 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75388e5-e6f3-440a-a9e0-834668d94887 admin admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:42.592826 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75388e5-e6f3-440a-a9e0-834668d94887 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:20:42.593138 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75388e5-e6f3-440a-a9e0-834668d94887 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:42.594746 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75388e5-e6f3-440a-a9e0-834668d94887 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:42.612331 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 809/1610] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Oct 12 00:20:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:20:42.620059 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75388e5-e6f3-440a-a9e0-834668d94887 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:42.627846 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75388e5-e6f3-440a-a9e0-834668d94887 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:42.632690 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9779daa-1605-4065-9922-35cabf0e6491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:42.633539 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9779daa-1605-4065-9922-35cabf0e6491 admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:42.633874 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9779daa-1605-4065-9922-35cabf0e6491 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:42.634210 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9779daa-1605-4065-9922-35cabf0e6491 admin admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:42.634903 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9779daa-1605-4065-9922-35cabf0e6491 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:42.636987 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9779daa-1605-4065-9922-35cabf0e6491 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:42.637723 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 802/1611] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:42 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:42.649152 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f25e0eb-407f-4334-87c4-617c46affa4c None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:42.650736 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-5f25e0eb-407f-4334-87c4-617c46affa4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:42.729588 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 810/1612] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:20:42 2022] POST /identity/v3/projects => generated 381 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 00:20:42.736865 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f25e0eb-407f-4334-87c4-617c46affa4c None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:42.737017 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9779daa-1605-4065-9922-35cabf0e6491 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:42.794142 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaf0c40-4aa6-4a03-ac8e-7561c5d4e68c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:42.795475 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf0c40-4aa6-4a03-ac8e-7561c5d4e68c service cinder] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:42.795883 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf0c40-4aa6-4a03-ac8e-7561c5d4e68c service cinder] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:42.797647 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf0c40-4aa6-4a03-ac8e-7561c5d4e68c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:42.810571 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9779daa-1605-4065-9922-35cabf0e6491 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:42.815957 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a504bda-f157-4d1b-a8b6-84dcfb57e0c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:42.816730 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a504bda-f157-4d1b-a8b6-84dcfb57e0c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:42.817098 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a504bda-f157-4d1b-a8b6-84dcfb57e0c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:42.817523 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a504bda-f157-4d1b-a8b6-84dcfb57e0c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:42.818248 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a504bda-f157-4d1b-a8b6-84dcfb57e0c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:42.820555 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a504bda-f157-4d1b-a8b6-84dcfb57e0c5 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:42.836923 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:42.884880 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaf0c40-4aa6-4a03-ac8e-7561c5d4e68c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:42.886574 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaf0c40-4aa6-4a03-ac8e-7561c5d4e68c service cinder] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:42.896836 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 803/1613] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:20:42 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 00:20:42.960917 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 811/1614] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:20:42 2022] POST /identity/v3/users => generated 291 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:20:42.964586 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaf0c40-4aa6-4a03-ac8e-7561c5d4e68c service cinder] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:42.988632 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaf0c40-4aa6-4a03-ac8e-7561c5d4e68c service cinder] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:42.994565 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0128e99-47bb-4922-bb8e-c4f600933ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:42.995716 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0128e99-47bb-4922-bb8e-c4f600933ff1 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:42.996005 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0128e99-47bb-4922-bb8e-c4f600933ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:42.996279 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0128e99-47bb-4922-bb8e-c4f600933ff1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:42.996971 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0128e99-47bb-4922-bb8e-c4f600933ff1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:42.998589 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0128e99-47bb-4922-bb8e-c4f600933ff1 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:43.019410 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 804/1615] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:20:42 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:20:43.019793 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a504bda-f157-4d1b-a8b6-84dcfb57e0c5 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:43.023224 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0128e99-47bb-4922-bb8e-c4f600933ff1 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:43.034157 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0128e99-47bb-4922-bb8e-c4f600933ff1 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:43.035437 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a504bda-f157-4d1b-a8b6-84dcfb57e0c5 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:43.039265 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827aa677-c58b-4195-ba1c-d0de3faff472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:43.039930 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827aa677-c58b-4195-ba1c-d0de3faff472 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:43.040215 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827aa677-c58b-4195-ba1c-d0de3faff472 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:43.040805 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827aa677-c58b-4195-ba1c-d0de3faff472 admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:43.041460 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827aa677-c58b-4195-ba1c-d0de3faff472 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:43.043399 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827aa677-c58b-4195-ba1c-d0de3faff472 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:43.044117 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefa7594-60c7-4356-8870-6914a181c576 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:43.044882 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefa7594-60c7-4356-8870-6914a181c576 service cinder] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:43.045250 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefa7594-60c7-4356-8870-6914a181c576 service cinder] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:43.045662 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefa7594-60c7-4356-8870-6914a181c576 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:43.059522 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 805/1616] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:20:43 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:20:43.060309 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827aa677-c58b-4195-ba1c-d0de3faff472 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:43.065924 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-fefa7594-60c7-4356-8870-6914a181c576 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:43.077889 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827aa677-c58b-4195-ba1c-d0de3faff472 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:43.084587 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b130f6-13de-4872-a040-6b48f239a6b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:43.089568 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b130f6-13de-4872-a040-6b48f239a6b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:43.089568 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b130f6-13de-4872-a040-6b48f239a6b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:43.089568 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b130f6-13de-4872-a040-6b48f239a6b6 admin admin] PATH_INFO: `/v3/projects/6b255ff81d314f1d8daa5515ae21af9b/users/53f8e91cced74f7fb64e77ea7149d5b7/roles/ef689f96442945e8a7bc8cbfee13d909` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:43.104304 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b130f6-13de-4872-a040-6b48f239a6b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b255ff81d314f1d8daa5515ae21af9b, user_id=53f8e91cced74f7fb64e77ea7149d5b7, role_id=ef689f96442945e8a7bc8cbfee13d909)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:43.107884 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b130f6-13de-4872-a040-6b48f239a6b6 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:43.175060 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefa7594-60c7-4356-8870-6914a181c576 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:43.176952 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefa7594-60c7-4356-8870-6914a181c576 service cinder] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:43.214907 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 812/1618] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:20:43 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:20:43.221308 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 806/1618] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:20:43 2022] PUT /identity/v3/projects/6b255ff81d314f1d8daa5515ae21af9b/users/53f8e91cced74f7fb64e77ea7149d5b7/roles/ef689f96442945e8a7bc8cbfee13d909 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:43.222759 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeaa438-5c1d-40f9-a9b8-8a5f6ad1ed0c None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:43.223172 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeaa438-5c1d-40f9-a9b8-8a5f6ad1ed0c None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:43.223429 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeaa438-5c1d-40f9-a9b8-8a5f6ad1ed0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:43.264250 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-abeaa438-5c1d-40f9-a9b8-8a5f6ad1ed0c None None] MFA Rules not processed for user `53f8e91cced74f7fb64e77ea7149d5b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:43.265211 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-abeaa438-5c1d-40f9-a9b8-8a5f6ad1ed0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:43.308440 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:43.333637 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:43.397859 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 813/1619] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:43 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:43.404645 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b130f6-13de-4872-a040-6b48f239a6b6 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:43.491092 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b130f6-13de-4872-a040-6b48f239a6b6 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:43.496555 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4187fd7f-f68e-4f90-a198-7b821e7a2270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:43.497169 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4187fd7f-f68e-4f90-a198-7b821e7a2270 admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:43.497447 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4187fd7f-f68e-4f90-a198-7b821e7a2270 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:43.497715 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4187fd7f-f68e-4f90-a198-7b821e7a2270 admin admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:43.498506 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4187fd7f-f68e-4f90-a198-7b821e7a2270 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:43.502242 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4187fd7f-f68e-4f90-a198-7b821e7a2270 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:43.605731 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 807/1620] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:20:43 2022] POST /identity/v3/projects => generated 383 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:20:43.606898 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeaa438-5c1d-40f9-a9b8-8a5f6ad1ed0c None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:43.619079 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeaa438-5c1d-40f9-a9b8-8a5f6ad1ed0c None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:43.677593 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a8eebb-6398-4931-b2e9-48576a5eb651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:43.678208 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a8eebb-6398-4931-b2e9-48576a5eb651 admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:43.678488 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a8eebb-6398-4931-b2e9-48576a5eb651 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:43.678758 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a8eebb-6398-4931-b2e9-48576a5eb651 admin admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:43.679465 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a8eebb-6398-4931-b2e9-48576a5eb651 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:43.682250 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a8eebb-6398-4931-b2e9-48576a5eb651 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:43.790968 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 814/1621] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:20:43 2022] POST /identity/v3/users => generated 298 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 00:20:43.792299 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4187fd7f-f68e-4f90-a198-7b821e7a2270 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:43.802993 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4187fd7f-f68e-4f90-a198-7b821e7a2270 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:43.808308 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12152af-0bbe-44cc-82cd-bb3a72c1d966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:43.812631 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12152af-0bbe-44cc-82cd-bb3a72c1d966 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:43.812631 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12152af-0bbe-44cc-82cd-bb3a72c1d966 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:43.812631 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12152af-0bbe-44cc-82cd-bb3a72c1d966 admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:43.812631 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12152af-0bbe-44cc-82cd-bb3a72c1d966 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:43.813163 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12152af-0bbe-44cc-82cd-bb3a72c1d966 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:43.828388 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 808/1622] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:20:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:20:43.829539 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a8eebb-6398-4931-b2e9-48576a5eb651 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:43.844985 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a8eebb-6398-4931-b2e9-48576a5eb651 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:43.855742 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13aaffa-ae87-4b5c-8827-5d33aac47aca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:43.856604 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13aaffa-ae87-4b5c-8827-5d33aac47aca admin admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:43.857310 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13aaffa-ae87-4b5c-8827-5d33aac47aca admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:43.857310 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13aaffa-ae87-4b5c-8827-5d33aac47aca admin admin] PATH_INFO: `/v3/projects/c1e6d093dd8d48b2b1e379baadda9199/users/cca2e17895cd4c42ad59b72d5211439a/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:43.875126 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13aaffa-ae87-4b5c-8827-5d33aac47aca admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1e6d093dd8d48b2b1e379baadda9199, user_id=cca2e17895cd4c42ad59b72d5211439a, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:43.877989 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13aaffa-ae87-4b5c-8827-5d33aac47aca admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:43.954733 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 815/1623] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:20:43 2022] PUT /identity/v3/projects/c1e6d093dd8d48b2b1e379baadda9199/users/cca2e17895cd4c42ad59b72d5211439a/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:43.959165 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12152af-0bbe-44cc-82cd-bb3a72c1d966 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:43.970483 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12152af-0bbe-44cc-82cd-bb3a72c1d966 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:44.054268 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e8392e-4b67-4789-9224-47b6b24508c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:44.054896 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e8392e-4b67-4789-9224-47b6b24508c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:44.055180 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e8392e-4b67-4789-9224-47b6b24508c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:44.055487 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e8392e-4b67-4789-9224-47b6b24508c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:44.056144 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e8392e-4b67-4789-9224-47b6b24508c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:44.057820 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e8392e-4b67-4789-9224-47b6b24508c2 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:44.073256 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 809/1624] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:20:43 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:20:44.074379 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13aaffa-ae87-4b5c-8827-5d33aac47aca admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:44.085428 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13aaffa-ae87-4b5c-8827-5d33aac47aca admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:44.090282 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eacb9f-f0ba-490b-ae00-0ef2e303b6a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:44.091169 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eacb9f-f0ba-490b-ae00-0ef2e303b6a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:44.091548 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eacb9f-f0ba-490b-ae00-0ef2e303b6a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:44.091906 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eacb9f-f0ba-490b-ae00-0ef2e303b6a3 admin admin] PATH_INFO: `/v3/projects/c1e6d093dd8d48b2b1e379baadda9199/users/cca2e17895cd4c42ad59b72d5211439a/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:44.094037 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6eacb9f-f0ba-490b-ae00-0ef2e303b6a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1e6d093dd8d48b2b1e379baadda9199, user_id=cca2e17895cd4c42ad59b72d5211439a, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:44.096802 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6eacb9f-f0ba-490b-ae00-0ef2e303b6a3 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:44.123036 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 816/1625] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:20:44 2022] PUT /identity/v3/projects/c1e6d093dd8d48b2b1e379baadda9199/users/cca2e17895cd4c42ad59b72d5211439a/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 00:20:44.132829 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f9fb4f-1652-49e0-a644-6e7843cfe4b0 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:44.133110 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f9fb4f-1652-49e0-a644-6e7843cfe4b0 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:44.133419 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f9fb4f-1652-49e0-a644-6e7843cfe4b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:44.181363 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-52f9fb4f-1652-49e0-a644-6e7843cfe4b0 None None] MFA Rules not processed for user `cca2e17895cd4c42ad59b72d5211439a`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:44.182601 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-52f9fb4f-1652-49e0-a644-6e7843cfe4b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:44.232961 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:44.257779 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 810/1626] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:44 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:44.269979 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eacb9f-f0ba-490b-ae00-0ef2e303b6a3 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:44.327880 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eacb9f-f0ba-490b-ae00-0ef2e303b6a3 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:44.331546 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d8433e-4593-4bbf-9a48-dd5e209166cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:44.332143 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d8433e-4593-4bbf-9a48-dd5e209166cf service cinder] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:44.332421 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d8433e-4593-4bbf-9a48-dd5e209166cf service cinder] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:44.332693 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d8433e-4593-4bbf-9a48-dd5e209166cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:44.342590 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d8433e-4593-4bbf-9a48-dd5e209166cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:44.344095 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d8433e-4593-4bbf-9a48-dd5e209166cf service cinder] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:44.354531 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 817/1627] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:20:44 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:20:44.494724 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f9fb4f-1652-49e0-a644-6e7843cfe4b0 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:44.503990 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f9fb4f-1652-49e0-a644-6e7843cfe4b0 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:44.507612 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28635b3-faf9-4213-b0f1-1f837ccbdf3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:44.511390 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28635b3-faf9-4213-b0f1-1f837ccbdf3c service cinder] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:44.511390 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28635b3-faf9-4213-b0f1-1f837ccbdf3c service cinder] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:44.511390 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28635b3-faf9-4213-b0f1-1f837ccbdf3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:44.609996 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28635b3-faf9-4213-b0f1-1f837ccbdf3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:44.611721 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28635b3-faf9-4213-b0f1-1f837ccbdf3c service cinder] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:44.623661 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 811/1628] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:20:44 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:20:44.816554 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:45.545978 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:45.611820 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d8433e-4593-4bbf-9a48-dd5e209166cf service cinder] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:45.677256 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d8433e-4593-4bbf-9a48-dd5e209166cf service cinder] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:45.681849 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6519ca-cfa9-4f2c-b813-642be826ee72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:45.682537 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6519ca-cfa9-4f2c-b813-642be826ee72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:45.682810 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6519ca-cfa9-4f2c-b813-642be826ee72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:45.683076 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6519ca-cfa9-4f2c-b813-642be826ee72 admin admin] PATH_INFO: `/v3/users/53f8e91cced74f7fb64e77ea7149d5b7` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:45.684910 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6519ca-cfa9-4f2c-b813-642be826ee72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53f8e91cced74f7fb64e77ea7149d5b7)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:45.688434 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6519ca-cfa9-4f2c-b813-642be826ee72 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:45.909647 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-ae6519ca-cfa9-4f2c-b813-642be826ee72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53f8e91cced74f7fb64e77ea7149d5b7'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:45.910979 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-ae6519ca-cfa9-4f2c-b813-642be826ee72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53f8e91cced74f7fb64e77ea7149d5b7'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:45.925483 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-ae6519ca-cfa9-4f2c-b813-642be826ee72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53f8e91cced74f7fb64e77ea7149d5b7'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:46.067711 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-ae6519ca-cfa9-4f2c-b813-642be826ee72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53f8e91cced74f7fb64e77ea7149d5b7'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:46.068350 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-ae6519ca-cfa9-4f2c-b813-642be826ee72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53f8e91cced74f7fb64e77ea7149d5b7'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:46.081377 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 818/1629] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:20:45 2022] DELETE /identity/v3/users/53f8e91cced74f7fb64e77ea7149d5b7 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:46.086148 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9021af-1454-4427-9402-82b9a3cb4a3f None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:46.086521 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9021af-1454-4427-9402-82b9a3cb4a3f None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:46.086861 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9021af-1454-4427-9402-82b9a3cb4a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:46.107959 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-1b9021af-1454-4427-9402-82b9a3cb4a3f None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:46.109246 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-1b9021af-1454-4427-9402-82b9a3cb4a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:46.203190 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 812/1630] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:46.204173 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:46.213217 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6519ca-cfa9-4f2c-b813-642be826ee72 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:46.214776 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-ae6519ca-cfa9-4f2c-b813-642be826ee72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:46.310863 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6519ca-cfa9-4f2c-b813-642be826ee72 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:46.320802 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ac61df-0af1-4a74-95e7-16553d836d14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:46.321770 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac61df-0af1-4a74-95e7-16553d836d14 service neutron] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:46.322236 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac61df-0af1-4a74-95e7-16553d836d14 service neutron] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:46.322600 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac61df-0af1-4a74-95e7-16553d836d14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:46.341380 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ac61df-0af1-4a74-95e7-16553d836d14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:46.343283 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ac61df-0af1-4a74-95e7-16553d836d14 service neutron] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:46.344490 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9021af-1454-4427-9402-82b9a3cb4a3f None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:46.346085 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-1b9021af-1454-4427-9402-82b9a3cb4a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:46.357494 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 819/1631] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:20:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:20:46.413913 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9021af-1454-4427-9402-82b9a3cb4a3f None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:46.418025 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9af522-e6ed-4042-928f-2fefea427f7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:46.418681 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9af522-e6ed-4042-928f-2fefea427f7f service cinder] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:46.418952 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9af522-e6ed-4042-928f-2fefea427f7f service cinder] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:46.419222 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9af522-e6ed-4042-928f-2fefea427f7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:46.421147 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-ba9af522-e6ed-4042-928f-2fefea427f7f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:46.493719 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9af522-e6ed-4042-928f-2fefea427f7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:46.495424 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9af522-e6ed-4042-928f-2fefea427f7f service cinder] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:46.508403 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 813/1632] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:20:46 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:20:46.832017 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ac61df-0af1-4a74-95e7-16553d836d14 service neutron] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:46.833876 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-53ac61df-0af1-4a74-95e7-16553d836d14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:46.834866 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9af522-e6ed-4042-928f-2fefea427f7f service cinder] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:46.836572 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-ba9af522-e6ed-4042-928f-2fefea427f7f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:46.847577 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9af522-e6ed-4042-928f-2fefea427f7f service cinder] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:46.852022 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1576968-353e-44f8-a4a9-743e23bf2490 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:46.853003 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ac61df-0af1-4a74-95e7-16553d836d14 service neutron] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:46.853391 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1576968-353e-44f8-a4a9-743e23bf2490 service cinder] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:46.853681 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1576968-353e-44f8-a4a9-743e23bf2490 service cinder] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:46.853959 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1576968-353e-44f8-a4a9-743e23bf2490 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:46.856392 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-a1576968-353e-44f8-a4a9-743e23bf2490 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:46.863954 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bd2c3c-91f3-48fe-81e7-e541756fbf75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:46.865299 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bd2c3c-91f3-48fe-81e7-e541756fbf75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:46.865811 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bd2c3c-91f3-48fe-81e7-e541756fbf75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:46.866086 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bd2c3c-91f3-48fe-81e7-e541756fbf75 admin admin] PATH_INFO: `/v3/users/349651cc16d74830a983e7e4cd05c199` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:46.867640 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bd2c3c-91f3-48fe-81e7-e541756fbf75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=349651cc16d74830a983e7e4cd05c199)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:46.869575 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bd2c3c-91f3-48fe-81e7-e541756fbf75 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:46.927137 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1576968-353e-44f8-a4a9-743e23bf2490 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:46.928656 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1576968-353e-44f8-a4a9-743e23bf2490 service cinder] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:46.938776 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 814/1633] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:20:46 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 00:20:46.939705 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1576968-353e-44f8-a4a9-743e23bf2490 service cinder] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:46.941186 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-a1576968-353e-44f8-a4a9-743e23bf2490 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:46.954967 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1576968-353e-44f8-a4a9-743e23bf2490 service cinder] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:46.962101 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599c6137-e9ff-457c-a711-e633273cc1e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:46.962745 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599c6137-e9ff-457c-a711-e633273cc1e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:46.963018 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599c6137-e9ff-457c-a711-e633273cc1e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:46.963286 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599c6137-e9ff-457c-a711-e633273cc1e7 admin admin] PATH_INFO: `/v3/projects/6b255ff81d314f1d8daa5515ae21af9b` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:46.964524 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599c6137-e9ff-457c-a711-e633273cc1e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b255ff81d314f1d8daa5515ae21af9b)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:46.966510 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599c6137-e9ff-457c-a711-e633273cc1e7 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:47.119885 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-599c6137-e9ff-457c-a711-e633273cc1e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b255ff81d314f1d8daa5515ae21af9b', 'request_id': 'req-599c6137-e9ff-457c-a711-e633273cc1e7'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:47.120488 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-31bd2c3c-91f3-48fe-81e7-e541756fbf75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '349651cc16d74830a983e7e4cd05c199'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:47.124302 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-31bd2c3c-91f3-48fe-81e7-e541756fbf75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '349651cc16d74830a983e7e4cd05c199'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:47.128576 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-599c6137-e9ff-457c-a711-e633273cc1e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b255ff81d314f1d8daa5515ae21af9b', 'request_id': 'req-599c6137-e9ff-457c-a711-e633273cc1e7'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:47.145820 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-31bd2c3c-91f3-48fe-81e7-e541756fbf75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '349651cc16d74830a983e7e4cd05c199'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:47.250406 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-599c6137-e9ff-457c-a711-e633273cc1e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b255ff81d314f1d8daa5515ae21af9b', 'request_id': 'req-599c6137-e9ff-457c-a711-e633273cc1e7'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:47.269619 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-599c6137-e9ff-457c-a711-e633273cc1e7 admin admin] The token cache is being invalidate because project 6b255ff81d314f1d8daa5515ae21af9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81746) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:20:47.269911 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-599c6137-e9ff-457c-a711-e633273cc1e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:47.270586 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-599c6137-e9ff-457c-a711-e633273cc1e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:47.272182 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 815/1634] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:20:46 2022] DELETE /identity/v3/projects/6b255ff81d314f1d8daa5515ae21af9b => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:47.276063 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599c6137-e9ff-457c-a711-e633273cc1e7 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:47.278235 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-599c6137-e9ff-457c-a711-e633273cc1e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:47.348540 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599c6137-e9ff-457c-a711-e633273cc1e7 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:47.353238 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885293a9-b6e5-4144-be0a-504b51ed2aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:47.353868 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885293a9-b6e5-4144-be0a-504b51ed2aa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:47.354144 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885293a9-b6e5-4144-be0a-504b51ed2aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:47.354412 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885293a9-b6e5-4144-be0a-504b51ed2aa6 admin admin] PATH_INFO: `/v3/users/cca2e17895cd4c42ad59b72d5211439a` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:47.355562 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885293a9-b6e5-4144-be0a-504b51ed2aa6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cca2e17895cd4c42ad59b72d5211439a)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:47.358137 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885293a9-b6e5-4144-be0a-504b51ed2aa6 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:47.369505 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-31bd2c3c-91f3-48fe-81e7-e541756fbf75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '349651cc16d74830a983e7e4cd05c199'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:47.370603 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-31bd2c3c-91f3-48fe-81e7-e541756fbf75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '349651cc16d74830a983e7e4cd05c199'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:47.388780 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 820/1635] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:20:46 2022] DELETE /identity/v3/users/349651cc16d74830a983e7e4cd05c199 => generated 0 bytes in 557 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:47.393774 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf8892-b3bb-4c43-a835-d9e169180158 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:47.394245 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf8892-b3bb-4c43-a835-d9e169180158 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:47.394667 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf8892-b3bb-4c43-a835-d9e169180158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:47.420956 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-5caf8892-b3bb-4c43-a835-d9e169180158 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:47.422203 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-5caf8892-b3bb-4c43-a835-d9e169180158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:47.492869 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 821/1636] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:47.505247 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caf8892-b3bb-4c43-a835-d9e169180158 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:47.506951 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-5caf8892-b3bb-4c43-a835-d9e169180158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:47.516586 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:47.578353 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-885293a9-b6e5-4144-be0a-504b51ed2aa6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cca2e17895cd4c42ad59b72d5211439a'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:47.579250 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-885293a9-b6e5-4144-be0a-504b51ed2aa6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cca2e17895cd4c42ad59b72d5211439a'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:47.599284 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caf8892-b3bb-4c43-a835-d9e169180158 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:47.602695 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-885293a9-b6e5-4144-be0a-504b51ed2aa6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cca2e17895cd4c42ad59b72d5211439a'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:47.603753 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ef3dc9-1b34-47ab-b4c3-4f95140421ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:47.604534 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef3dc9-1b34-47ab-b4c3-4f95140421ca service neutron] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:47.604961 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef3dc9-1b34-47ab-b4c3-4f95140421ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:47.605231 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef3dc9-1b34-47ab-b4c3-4f95140421ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:47.608658 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-a3ef3dc9-1b34-47ab-b4c3-4f95140421ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:47.621832 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ef3dc9-1b34-47ab-b4c3-4f95140421ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:47.623394 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ef3dc9-1b34-47ab-b4c3-4f95140421ca service neutron] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:47.633267 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 822/1637] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:20:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 00:20:47.740744 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-885293a9-b6e5-4144-be0a-504b51ed2aa6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cca2e17895cd4c42ad59b72d5211439a'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:47.741395 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-885293a9-b6e5-4144-be0a-504b51ed2aa6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cca2e17895cd4c42ad59b72d5211439a'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:47.768343 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 816/1638] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:20:47 2022] DELETE /identity/v3/users/cca2e17895cd4c42ad59b72d5211439a => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 00:20:48.196718 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ef3dc9-1b34-47ab-b4c3-4f95140421ca service neutron] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:48.199219 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-a3ef3dc9-1b34-47ab-b4c3-4f95140421ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:48.219804 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ef3dc9-1b34-47ab-b4c3-4f95140421ca service neutron] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:48.287747 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eb10cf-e66b-4ba4-9548-7cd7af56909d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:48.288568 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eb10cf-e66b-4ba4-9548-7cd7af56909d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:48.288897 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eb10cf-e66b-4ba4-9548-7cd7af56909d admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:48.289424 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eb10cf-e66b-4ba4-9548-7cd7af56909d admin admin] PATH_INFO: `/v3/projects/035b882afc6941fa94810b16ca6fef2c` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:48.291091 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28eb10cf-e66b-4ba4-9548-7cd7af56909d admin admin] RBAC: Authorizing `identity:delete_project(project_id=035b882afc6941fa94810b16ca6fef2c)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:48.293574 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28eb10cf-e66b-4ba4-9548-7cd7af56909d admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:48.311472 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:48.385110 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885293a9-b6e5-4144-be0a-504b51ed2aa6 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:48.396344 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-885293a9-b6e5-4144-be0a-504b51ed2aa6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:48.417524 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885293a9-b6e5-4144-be0a-504b51ed2aa6 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:48.442804 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b0a9e6-747f-49b0-bb0e-82e08afd9013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:48.443487 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b0a9e6-747f-49b0-bb0e-82e08afd9013 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:48.443770 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b0a9e6-747f-49b0-bb0e-82e08afd9013 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:48.444047 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b0a9e6-747f-49b0-bb0e-82e08afd9013 admin admin] PATH_INFO: `/v3/projects/c1e6d093dd8d48b2b1e379baadda9199` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:48.448079 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b0a9e6-747f-49b0-bb0e-82e08afd9013 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1e6d093dd8d48b2b1e379baadda9199)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:48.450578 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b0a9e6-747f-49b0-bb0e-82e08afd9013 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:48.490053 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-28eb10cf-e66b-4ba4-9548-7cd7af56909d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '035b882afc6941fa94810b16ca6fef2c', 'request_id': 'req-28eb10cf-e66b-4ba4-9548-7cd7af56909d'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:48.500704 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-28eb10cf-e66b-4ba4-9548-7cd7af56909d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '035b882afc6941fa94810b16ca6fef2c', 'request_id': 'req-28eb10cf-e66b-4ba4-9548-7cd7af56909d'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:48.578988 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-28eb10cf-e66b-4ba4-9548-7cd7af56909d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '035b882afc6941fa94810b16ca6fef2c', 'request_id': 'req-28eb10cf-e66b-4ba4-9548-7cd7af56909d'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:48.591232 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-28eb10cf-e66b-4ba4-9548-7cd7af56909d admin admin] The token cache is being invalidate because project 035b882afc6941fa94810b16ca6fef2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81747) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:20:48.591573 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-28eb10cf-e66b-4ba4-9548-7cd7af56909d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:48.592222 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-28eb10cf-e66b-4ba4-9548-7cd7af56909d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:48.593894 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 823/1639] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:20:48 2022] DELETE /identity/v3/projects/035b882afc6941fa94810b16ca6fef2c => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:48.597695 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-67b0a9e6-747f-49b0-bb0e-82e08afd9013 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1e6d093dd8d48b2b1e379baadda9199', 'request_id': 'req-67b0a9e6-747f-49b0-bb0e-82e08afd9013'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:48.597870 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eb10cf-e66b-4ba4-9548-7cd7af56909d admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:48.599476 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-28eb10cf-e66b-4ba4-9548-7cd7af56909d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:48.604271 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-67b0a9e6-747f-49b0-bb0e-82e08afd9013 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1e6d093dd8d48b2b1e379baadda9199', 'request_id': 'req-67b0a9e6-747f-49b0-bb0e-82e08afd9013'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:48.662478 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eb10cf-e66b-4ba4-9548-7cd7af56909d admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:48.713760 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-67b0a9e6-747f-49b0-bb0e-82e08afd9013 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1e6d093dd8d48b2b1e379baadda9199', 'request_id': 'req-67b0a9e6-747f-49b0-bb0e-82e08afd9013'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:48.727325 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-67b0a9e6-747f-49b0-bb0e-82e08afd9013 admin admin] The token cache is being invalidate because project c1e6d093dd8d48b2b1e379baadda9199 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81746) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:20:48.728113 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-67b0a9e6-747f-49b0-bb0e-82e08afd9013 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:48.728810 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-67b0a9e6-747f-49b0-bb0e-82e08afd9013 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:48.731414 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 817/1640] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:20:48 2022] DELETE /identity/v3/projects/c1e6d093dd8d48b2b1e379baadda9199 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:48.738079 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb11e40-40b3-4bd1-bf03-6f1b1e7dcc55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:48.739190 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb11e40-40b3-4bd1-bf03-6f1b1e7dcc55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:48.739864 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb11e40-40b3-4bd1-bf03-6f1b1e7dcc55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:48.741210 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb11e40-40b3-4bd1-bf03-6f1b1e7dcc55 admin admin] PATH_INFO: `/v3/users/9b744fdcd677444e89933e7193c4b230` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:48.742704 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb11e40-40b3-4bd1-bf03-6f1b1e7dcc55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b744fdcd677444e89933e7193c4b230)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:48.745367 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb11e40-40b3-4bd1-bf03-6f1b1e7dcc55 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:48.750533 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1332b150-2384-471e-80a2-e2dc14fc6af0 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:48.750890 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1332b150-2384-471e-80a2-e2dc14fc6af0 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:48.751169 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1332b150-2384-471e-80a2-e2dc14fc6af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:48.779902 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-1332b150-2384-471e-80a2-e2dc14fc6af0 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:48.781306 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-1332b150-2384-471e-80a2-e2dc14fc6af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:48.796669 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 818/1641] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 00:20:48.798308 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd287de4-7cc0-41fe-91d9-0c3c3e23c5c4 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:48.801494 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd287de4-7cc0-41fe-91d9-0c3c3e23c5c4 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:48.801494 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd287de4-7cc0-41fe-91d9-0c3c3e23c5c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:48.829503 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-bd287de4-7cc0-41fe-91d9-0c3c3e23c5c4 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:48.830764 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-bd287de4-7cc0-41fe-91d9-0c3c3e23c5c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:48.839315 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 819/1642] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 00:20:48.843966 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd287de4-7cc0-41fe-91d9-0c3c3e23c5c4 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:48.853283 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd287de4-7cc0-41fe-91d9-0c3c3e23c5c4 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:48.858466 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e84b6a-f648-4743-861d-4c7d656003c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:48.859072 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e84b6a-f648-4743-861d-4c7d656003c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:48.859349 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e84b6a-f648-4743-861d-4c7d656003c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:48.859655 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e84b6a-f648-4743-861d-4c7d656003c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:48.860420 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e84b6a-f648-4743-861d-4c7d656003c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:20:48.860722 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e84b6a-f648-4743-861d-4c7d656003c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:48.862412 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e84b6a-f648-4743-861d-4c7d656003c7 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:48.882664 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 820/1643] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Oct 12 00:20:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 00:20:48.888765 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e84b6a-f648-4743-861d-4c7d656003c7 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:48.899068 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e84b6a-f648-4743-861d-4c7d656003c7 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:48.904972 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2463ee-b586-42ee-b0da-72e1e24dc863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:48.910532 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2463ee-b586-42ee-b0da-72e1e24dc863 admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:48.910532 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2463ee-b586-42ee-b0da-72e1e24dc863 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:48.910532 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2463ee-b586-42ee-b0da-72e1e24dc863 admin admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:48.910532 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2463ee-b586-42ee-b0da-72e1e24dc863 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:48.910532 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2463ee-b586-42ee-b0da-72e1e24dc863 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:48.970998 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-8eb11e40-40b3-4bd1-bf03-6f1b1e7dcc55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b744fdcd677444e89933e7193c4b230'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:48.971760 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-8eb11e40-40b3-4bd1-bf03-6f1b1e7dcc55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b744fdcd677444e89933e7193c4b230'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:48.985894 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-8eb11e40-40b3-4bd1-bf03-6f1b1e7dcc55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b744fdcd677444e89933e7193c4b230'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:49.016865 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 821/1644] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:20:48 2022] POST /identity/v3/projects => generated 375 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 00:20:49.024482 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2463ee-b586-42ee-b0da-72e1e24dc863 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:49.035310 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-de2463ee-b586-42ee-b0da-72e1e24dc863 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:49.119330 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-8eb11e40-40b3-4bd1-bf03-6f1b1e7dcc55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b744fdcd677444e89933e7193c4b230'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:49.119972 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-8eb11e40-40b3-4bd1-bf03-6f1b1e7dcc55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b744fdcd677444e89933e7193c4b230'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:49.139091 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 824/1645] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:20:48 2022] DELETE /identity/v3/users/9b744fdcd677444e89933e7193c4b230 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 00:20:49.146074 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2463ee-b586-42ee-b0da-72e1e24dc863 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:49.217471 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1dd5b4-e406-4c32-aac3-82d0a6336487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:49.219051 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1dd5b4-e406-4c32-aac3-82d0a6336487 admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:49.219051 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1dd5b4-e406-4c32-aac3-82d0a6336487 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:49.219051 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1dd5b4-e406-4c32-aac3-82d0a6336487 admin admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:49.219673 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1dd5b4-e406-4c32-aac3-82d0a6336487 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:49.221928 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1dd5b4-e406-4c32-aac3-82d0a6336487 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:49.313108 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 822/1646] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:20:49 2022] POST /identity/v3/users => generated 288 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:20:49.317139 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb11e40-40b3-4bd1-bf03-6f1b1e7dcc55 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:49.329889 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb11e40-40b3-4bd1-bf03-6f1b1e7dcc55 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:49.334992 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af84f5b-5473-4165-b996-c28b47df9cf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:49.335653 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af84f5b-5473-4165-b996-c28b47df9cf3 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:49.335977 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af84f5b-5473-4165-b996-c28b47df9cf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:49.336391 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af84f5b-5473-4165-b996-c28b47df9cf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:49.337507 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af84f5b-5473-4165-b996-c28b47df9cf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:49.341846 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af84f5b-5473-4165-b996-c28b47df9cf3 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:49.359989 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 825/1647] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:20:49 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:20:49.360850 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1dd5b4-e406-4c32-aac3-82d0a6336487 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:49.371922 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1dd5b4-e406-4c32-aac3-82d0a6336487 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:49.377774 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363c10a1-0e8f-4b99-bdee-7dc235bbaf3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:49.378378 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c10a1-0e8f-4b99-bdee-7dc235bbaf3a admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:49.378652 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c10a1-0e8f-4b99-bdee-7dc235bbaf3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:49.378918 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c10a1-0e8f-4b99-bdee-7dc235bbaf3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:49.379644 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363c10a1-0e8f-4b99-bdee-7dc235bbaf3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:49.381238 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363c10a1-0e8f-4b99-bdee-7dc235bbaf3a admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:49.392584 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 823/1648] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:20:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:20:49.396663 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af84f5b-5473-4165-b996-c28b47df9cf3 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:49.406949 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af84f5b-5473-4165-b996-c28b47df9cf3 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:49.411402 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a948507b-7b09-4ff9-a77c-6677669b12a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:49.412066 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948507b-7b09-4ff9-a77c-6677669b12a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:49.412346 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948507b-7b09-4ff9-a77c-6677669b12a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:49.412611 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948507b-7b09-4ff9-a77c-6677669b12a2 admin admin] PATH_INFO: `/v3/projects/879b48942542406ba8df21971f4174cb/users/77a415a0c886456695ecb014b0ffbdd7/roles/ef689f96442945e8a7bc8cbfee13d909` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:49.427538 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a948507b-7b09-4ff9-a77c-6677669b12a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=879b48942542406ba8df21971f4174cb, user_id=77a415a0c886456695ecb014b0ffbdd7, role_id=ef689f96442945e8a7bc8cbfee13d909)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:49.430180 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a948507b-7b09-4ff9-a77c-6677669b12a2 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:49.529831 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 826/1649] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:20:49 2022] PUT /identity/v3/projects/879b48942542406ba8df21971f4174cb/users/77a415a0c886456695ecb014b0ffbdd7/roles/ef689f96442945e8a7bc8cbfee13d909 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:49.544114 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee317f9-8f28-43a0-b8ad-a21adc07886a None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:49.544555 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee317f9-8f28-43a0-b8ad-a21adc07886a None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:49.544848 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee317f9-8f28-43a0-b8ad-a21adc07886a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:49.559863 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a948507b-7b09-4ff9-a77c-6677669b12a2 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:49.562038 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-a948507b-7b09-4ff9-a77c-6677669b12a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:49.595256 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-dee317f9-8f28-43a0-b8ad-a21adc07886a None None] MFA Rules not processed for user `77a415a0c886456695ecb014b0ffbdd7`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:49.596839 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-dee317f9-8f28-43a0-b8ad-a21adc07886a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:49.664932 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a948507b-7b09-4ff9-a77c-6677669b12a2 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:49.670241 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65325b3a-4d1e-4809-8f34-d1edc76cbe6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:49.670899 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65325b3a-4d1e-4809-8f34-d1edc76cbe6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:49.671178 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65325b3a-4d1e-4809-8f34-d1edc76cbe6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:49.671523 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65325b3a-4d1e-4809-8f34-d1edc76cbe6c admin admin] PATH_INFO: `/v3/projects/94e0e8b6283f47de8d979032c8098d30` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:49.672691 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65325b3a-4d1e-4809-8f34-d1edc76cbe6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=94e0e8b6283f47de8d979032c8098d30)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:49.674877 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65325b3a-4d1e-4809-8f34-d1edc76cbe6c admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:49.676738 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 824/1650] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:49.681310 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee317f9-8f28-43a0-b8ad-a21adc07886a None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:49.696454 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee317f9-8f28-43a0-b8ad-a21adc07886a None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:49.701450 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42966195-e776-4bdd-819c-c1225065dfb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:49.702065 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42966195-e776-4bdd-819c-c1225065dfb7 admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:49.702347 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42966195-e776-4bdd-819c-c1225065dfb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:49.702622 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42966195-e776-4bdd-819c-c1225065dfb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:49.703257 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42966195-e776-4bdd-819c-c1225065dfb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:49.705209 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42966195-e776-4bdd-819c-c1225065dfb7 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:49.859043 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-65325b3a-4d1e-4809-8f34-d1edc76cbe6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94e0e8b6283f47de8d979032c8098d30', 'request_id': 'req-65325b3a-4d1e-4809-8f34-d1edc76cbe6c'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:49.865257 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-65325b3a-4d1e-4809-8f34-d1edc76cbe6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94e0e8b6283f47de8d979032c8098d30', 'request_id': 'req-65325b3a-4d1e-4809-8f34-d1edc76cbe6c'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:49.888245 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:49.899752 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 825/1651] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:20:49 2022] POST /identity/v3/projects => generated 375 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 00:20:49.900844 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42966195-e776-4bdd-819c-c1225065dfb7 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:49.956187 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-65325b3a-4d1e-4809-8f34-d1edc76cbe6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94e0e8b6283f47de8d979032c8098d30', 'request_id': 'req-65325b3a-4d1e-4809-8f34-d1edc76cbe6c'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:49.971006 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42966195-e776-4bdd-819c-c1225065dfb7 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:49.972246 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-65325b3a-4d1e-4809-8f34-d1edc76cbe6c admin admin] The token cache is being invalidate because project 94e0e8b6283f47de8d979032c8098d30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81747) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:20:49.972772 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-65325b3a-4d1e-4809-8f34-d1edc76cbe6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:49.973780 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-65325b3a-4d1e-4809-8f34-d1edc76cbe6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:49.975866 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 827/1652] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:20:49 2022] DELETE /identity/v3/projects/94e0e8b6283f47de8d979032c8098d30 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:49.979410 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfac751-2bbc-4e34-82e4-34f3aaaaacf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:49.980068 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfac751-2bbc-4e34-82e4-34f3aaaaacf5 admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:49.980352 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfac751-2bbc-4e34-82e4-34f3aaaaacf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:49.981143 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfac751-2bbc-4e34-82e4-34f3aaaaacf5 admin admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:49.981374 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd13853-9520-46a0-ac21-522f60fdf497 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:49.981673 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd13853-9520-46a0-ac21-522f60fdf497 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:49.981773 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfac751-2bbc-4e34-82e4-34f3aaaaacf5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:49.981937 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd13853-9520-46a0-ac21-522f60fdf497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:49.985692 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfac751-2bbc-4e34-82e4-34f3aaaaacf5 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:49.987954 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:50.003766 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-8fd13853-9520-46a0-ac21-522f60fdf497 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:50.004984 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-8fd13853-9520-46a0-ac21-522f60fdf497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:50.013086 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 828/1653] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 00:20:50.021296 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f0e8de-b2c8-4aad-b15c-93ad102224b0 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:50.021585 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f0e8de-b2c8-4aad-b15c-93ad102224b0 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:50.026356 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f0e8de-b2c8-4aad-b15c-93ad102224b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:50.041199 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-c7f0e8de-b2c8-4aad-b15c-93ad102224b0 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:50.042399 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-c7f0e8de-b2c8-4aad-b15c-93ad102224b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:50.050861 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 829/1654] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 00:20:50.054983 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f0e8de-b2c8-4aad-b15c-93ad102224b0 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:50.064576 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f0e8de-b2c8-4aad-b15c-93ad102224b0 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:50.069202 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc8ba3b-e2b1-4f84-977f-e9ed62bacc75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:50.069789 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc8ba3b-e2b1-4f84-977f-e9ed62bacc75 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:50.070063 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc8ba3b-e2b1-4f84-977f-e9ed62bacc75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:50.070340 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc8ba3b-e2b1-4f84-977f-e9ed62bacc75 admin admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:50.070970 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc8ba3b-e2b1-4f84-977f-e9ed62bacc75 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:20:50.071268 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc8ba3b-e2b1-4f84-977f-e9ed62bacc75 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:50.072885 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc8ba3b-e2b1-4f84-977f-e9ed62bacc75 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:50.084630 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 830/1655] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Oct 12 00:20:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 00:20:50.088885 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc8ba3b-e2b1-4f84-977f-e9ed62bacc75 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:50.099674 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc8ba3b-e2b1-4f84-977f-e9ed62bacc75 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:50.106503 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fc6159-07ea-4559-af00-83cd668ff759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:50.107124 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fc6159-07ea-4559-af00-83cd668ff759 admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:50.107428 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fc6159-07ea-4559-af00-83cd668ff759 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:50.107700 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fc6159-07ea-4559-af00-83cd668ff759 admin admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:50.108347 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fc6159-07ea-4559-af00-83cd668ff759 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:50.110279 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fc6159-07ea-4559-af00-83cd668ff759 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:50.111236 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 826/1656] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:20:49 2022] POST /identity/v3/users => generated 288 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:20:50.115732 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfac751-2bbc-4e34-82e4-34f3aaaaacf5 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:50.123487 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-1bfac751-2bbc-4e34-82e4-34f3aaaaacf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:50.139658 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfac751-2bbc-4e34-82e4-34f3aaaaacf5 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:50.145111 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad414a9e-c156-4d21-87cb-05b6a61d49a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:50.145580 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad414a9e-c156-4d21-87cb-05b6a61d49a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:50.145810 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad414a9e-c156-4d21-87cb-05b6a61d49a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:50.146168 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad414a9e-c156-4d21-87cb-05b6a61d49a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:50.146815 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad414a9e-c156-4d21-87cb-05b6a61d49a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:50.148529 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad414a9e-c156-4d21-87cb-05b6a61d49a6 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:50.158814 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 827/1657] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:20:50 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:20:50.162660 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad414a9e-c156-4d21-87cb-05b6a61d49a6 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:50.173113 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad414a9e-c156-4d21-87cb-05b6a61d49a6 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:50.177933 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9098f730-cbe8-410b-b287-22f82a750b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:50.178535 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9098f730-cbe8-410b-b287-22f82a750b38 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:50.178816 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9098f730-cbe8-410b-b287-22f82a750b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:50.179085 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9098f730-cbe8-410b-b287-22f82a750b38 admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:50.179748 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9098f730-cbe8-410b-b287-22f82a750b38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:50.181335 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9098f730-cbe8-410b-b287-22f82a750b38 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:50.196142 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 828/1658] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:20:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 00:20:50.207393 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 831/1659] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:20:50 2022] POST /identity/v3/projects => generated 373 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 00:20:50.208426 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9098f730-cbe8-410b-b287-22f82a750b38 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:50.211419 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fc6159-07ea-4559-af00-83cd668ff759 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:50.223064 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9098f730-cbe8-410b-b287-22f82a750b38 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:50.312273 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fc6159-07ea-4559-af00-83cd668ff759 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:50.317153 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67611d0-3c56-4dd3-8725-af06dd7dfa91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:50.317803 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67611d0-3c56-4dd3-8725-af06dd7dfa91 admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:50.322112 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67611d0-3c56-4dd3-8725-af06dd7dfa91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:50.322112 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67611d0-3c56-4dd3-8725-af06dd7dfa91 admin admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:50.322112 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67611d0-3c56-4dd3-8725-af06dd7dfa91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:50.322112 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67611d0-3c56-4dd3-8725-af06dd7dfa91 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:50.329484 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e896ef9-c03b-486e-b32e-99ebaaaac40a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:50.330911 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e896ef9-c03b-486e-b32e-99ebaaaac40a admin admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:50.330911 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e896ef9-c03b-486e-b32e-99ebaaaac40a admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:50.331351 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e896ef9-c03b-486e-b32e-99ebaaaac40a admin admin] PATH_INFO: `/v3/projects/5028d039086546f3a267f246a6734656/users/2fd2cf0fe8c54f5b9398bfcd8942bb0f/roles/ef689f96442945e8a7bc8cbfee13d909` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:50.350954 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e896ef9-c03b-486e-b32e-99ebaaaac40a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5028d039086546f3a267f246a6734656, user_id=2fd2cf0fe8c54f5b9398bfcd8942bb0f, role_id=ef689f96442945e8a7bc8cbfee13d909)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:50.353929 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e896ef9-c03b-486e-b32e-99ebaaaac40a admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:50.421369 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 832/1660] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:20:50 2022] POST /identity/v3/users => generated 287 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:20:50.422128 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67611d0-3c56-4dd3-8725-af06dd7dfa91 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:50.422994 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 829/1661] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:20:50 2022] PUT /identity/v3/projects/5028d039086546f3a267f246a6734656/users/2fd2cf0fe8c54f5b9398bfcd8942bb0f/roles/ef689f96442945e8a7bc8cbfee13d909 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:50.435428 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab65616-d759-4140-a713-2694b3c7cee5 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:50.435742 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab65616-d759-4140-a713-2694b3c7cee5 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:50.436003 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab65616-d759-4140-a713-2694b3c7cee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:50.440382 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:50.446854 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67611d0-3c56-4dd3-8725-af06dd7dfa91 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:50.493060 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-fab65616-d759-4140-a713-2694b3c7cee5 None None] MFA Rules not processed for user `2fd2cf0fe8c54f5b9398bfcd8942bb0f`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:50.494569 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-fab65616-d759-4140-a713-2694b3c7cee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:50.537034 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6f48bf-c165-4ba5-a1e5-987a51c93008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:50.537858 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f48bf-c165-4ba5-a1e5-987a51c93008 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:50.539235 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f48bf-c165-4ba5-a1e5-987a51c93008 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:50.539235 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f48bf-c165-4ba5-a1e5-987a51c93008 admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:50.539884 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6f48bf-c165-4ba5-a1e5-987a51c93008 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:50.541525 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6f48bf-c165-4ba5-a1e5-987a51c93008 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:50.556830 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 833/1662] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:20:50 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:20:50.560994 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6f48bf-c165-4ba5-a1e5-987a51c93008 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:50.581283 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6f48bf-c165-4ba5-a1e5-987a51c93008 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:50.588221 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 830/1663] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:50.588356 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea226e4-85e3-427f-a79a-bfafdee57df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:50.588704 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea226e4-85e3-427f-a79a-bfafdee57df8 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:50.589013 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea226e4-85e3-427f-a79a-bfafdee57df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:50.589289 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea226e4-85e3-427f-a79a-bfafdee57df8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:50.590463 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea226e4-85e3-427f-a79a-bfafdee57df8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:50.592165 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea226e4-85e3-427f-a79a-bfafdee57df8 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:50.593075 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab65616-d759-4140-a713-2694b3c7cee5 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:50.603240 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 834/1664] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:20:50 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:20:50.603604 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab65616-d759-4140-a713-2694b3c7cee5 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:50.609281 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513f7e27-b2e5-4213-aef0-645a38ecf155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:50.609885 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f7e27-b2e5-4213-aef0-645a38ecf155 admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:50.610165 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f7e27-b2e5-4213-aef0-645a38ecf155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:50.610439 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f7e27-b2e5-4213-aef0-645a38ecf155 admin admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:50.611074 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513f7e27-b2e5-4213-aef0-645a38ecf155 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:50.611648 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea226e4-85e3-427f-a79a-bfafdee57df8 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:50.613080 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513f7e27-b2e5-4213-aef0-645a38ecf155 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:50.621406 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea226e4-85e3-427f-a79a-bfafdee57df8 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:50.626422 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c474eebb-75ff-4f1a-9a07-00506337bd49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:50.631615 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c474eebb-75ff-4f1a-9a07-00506337bd49 admin admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:50.631615 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c474eebb-75ff-4f1a-9a07-00506337bd49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:50.631615 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c474eebb-75ff-4f1a-9a07-00506337bd49 admin admin] PATH_INFO: `/v3/projects/063a92e4b88f43b88a05dcfda7e19d89/users/a491001660e549db85a151e77711797c/roles/ef689f96442945e8a7bc8cbfee13d909` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:50.642480 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c474eebb-75ff-4f1a-9a07-00506337bd49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=063a92e4b88f43b88a05dcfda7e19d89, user_id=a491001660e549db85a151e77711797c, role_id=ef689f96442945e8a7bc8cbfee13d909)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:50.645185 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c474eebb-75ff-4f1a-9a07-00506337bd49 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:50.771170 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 835/1665] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:20:50 2022] PUT /identity/v3/projects/063a92e4b88f43b88a05dcfda7e19d89/users/a491001660e549db85a151e77711797c/roles/ef689f96442945e8a7bc8cbfee13d909 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 00:20:50.775005 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 831/1666] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:20:50 2022] POST /identity/v3/projects => generated 377 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 00:20:50.776029 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513f7e27-b2e5-4213-aef0-645a38ecf155 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:50.776150 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f408cd74-3f6b-4f1f-ad7e-d7f389e662ff None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:50.776451 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f408cd74-3f6b-4f1f-ad7e-d7f389e662ff None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:50.776724 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f408cd74-3f6b-4f1f-ad7e-d7f389e662ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:50.797612 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-f408cd74-3f6b-4f1f-ad7e-d7f389e662ff None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:50.800328 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-f408cd74-3f6b-4f1f-ad7e-d7f389e662ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:50.842271 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513f7e27-b2e5-4213-aef0-645a38ecf155 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:50.850687 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97658b40-0783-4baa-8bb8-05d74602ce28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:50.851736 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97658b40-0783-4baa-8bb8-05d74602ce28 admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:50.852018 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97658b40-0783-4baa-8bb8-05d74602ce28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:50.852339 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97658b40-0783-4baa-8bb8-05d74602ce28 admin admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:50.852982 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97658b40-0783-4baa-8bb8-05d74602ce28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:50.855001 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97658b40-0783-4baa-8bb8-05d74602ce28 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:50.870505 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 836/1667] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:50.903725 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f408cd74-3f6b-4f1f-ad7e-d7f389e662ff None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:50.906093 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-f408cd74-3f6b-4f1f-ad7e-d7f389e662ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:50.976449 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f408cd74-3f6b-4f1f-ad7e-d7f389e662ff None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:50.980947 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcccfa71-f8f3-4f03-8001-841662ab4ff6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:50.981620 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcccfa71-f8f3-4f03-8001-841662ab4ff6 service neutron] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:50.981899 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcccfa71-f8f3-4f03-8001-841662ab4ff6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:50.982227 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcccfa71-f8f3-4f03-8001-841662ab4ff6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:50.989461 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 832/1668] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:20:50 2022] POST /identity/v3/users => generated 295 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 00:20:50.992706 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcccfa71-f8f3-4f03-8001-841662ab4ff6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:50.993724 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97658b40-0783-4baa-8bb8-05d74602ce28 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:50.994147 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcccfa71-f8f3-4f03-8001-841662ab4ff6 service neutron] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:51.004294 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97658b40-0783-4baa-8bb8-05d74602ce28 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:51.004478 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 837/1669] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:20:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 00:20:51.009239 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c84f08-4dd8-4a02-b05f-783d99b7889a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:51.009883 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c84f08-4dd8-4a02-b05f-783d99b7889a admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:51.010159 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c84f08-4dd8-4a02-b05f-783d99b7889a admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:51.010425 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c84f08-4dd8-4a02-b05f-783d99b7889a admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:51.011061 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c84f08-4dd8-4a02-b05f-783d99b7889a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:51.013235 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c84f08-4dd8-4a02-b05f-783d99b7889a admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:51.026792 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 833/1670] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:20:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:20:51.028598 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcccfa71-f8f3-4f03-8001-841662ab4ff6 service neutron] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:51.038050 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcccfa71-f8f3-4f03-8001-841662ab4ff6 service neutron] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:51.042706 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcaa01e-15ed-4633-ba0f-ebf72212e613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:51.043504 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcaa01e-15ed-4633-ba0f-ebf72212e613 admin admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:51.043777 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcaa01e-15ed-4633-ba0f-ebf72212e613 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:51.044097 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcaa01e-15ed-4633-ba0f-ebf72212e613 admin admin] PATH_INFO: `/v3/projects/9aef5295bd4949a8b2cb6478e578e569/users/bd3967ff06eb4f18bdebd396dbfa2d65/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:51.059784 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcaa01e-15ed-4633-ba0f-ebf72212e613 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9aef5295bd4949a8b2cb6478e578e569, user_id=bd3967ff06eb4f18bdebd396dbfa2d65, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:51.062564 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcaa01e-15ed-4633-ba0f-ebf72212e613 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:51.134501 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 838/1671] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:20:51 2022] PUT /identity/v3/projects/9aef5295bd4949a8b2cb6478e578e569/users/bd3967ff06eb4f18bdebd396dbfa2d65/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 00:20:51.136148 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c84f08-4dd8-4a02-b05f-783d99b7889a admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:51.147724 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c84f08-4dd8-4a02-b05f-783d99b7889a admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:51.221887 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03389bc3-9a46-4c96-ae87-eeefee1b8a0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:51.222517 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03389bc3-9a46-4c96-ae87-eeefee1b8a0a admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:51.222798 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03389bc3-9a46-4c96-ae87-eeefee1b8a0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:51.223072 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03389bc3-9a46-4c96-ae87-eeefee1b8a0a admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:51.223766 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03389bc3-9a46-4c96-ae87-eeefee1b8a0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:51.225584 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03389bc3-9a46-4c96-ae87-eeefee1b8a0a admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:51.239564 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 834/1672] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:20:51 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:20:51.241370 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcaa01e-15ed-4633-ba0f-ebf72212e613 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:51.254386 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcaa01e-15ed-4633-ba0f-ebf72212e613 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:51.258978 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2ef3e6-d16e-4f44-9415-a614a55582c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:51.259672 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2ef3e6-d16e-4f44-9415-a614a55582c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:51.259951 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2ef3e6-d16e-4f44-9415-a614a55582c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:51.260221 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2ef3e6-d16e-4f44-9415-a614a55582c3 admin admin] PATH_INFO: `/v3/projects/9aef5295bd4949a8b2cb6478e578e569/users/bd3967ff06eb4f18bdebd396dbfa2d65/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:51.262768 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2ef3e6-d16e-4f44-9415-a614a55582c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9aef5295bd4949a8b2cb6478e578e569, user_id=bd3967ff06eb4f18bdebd396dbfa2d65, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:51.265678 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2ef3e6-d16e-4f44-9415-a614a55582c3 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:51.278930 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 839/1673] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:20:51 2022] PUT /identity/v3/projects/9aef5295bd4949a8b2cb6478e578e569/users/bd3967ff06eb4f18bdebd396dbfa2d65/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 00:20:51.288834 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff4fa80-fa76-424e-93e9-0fd553ec34f3 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:51.289120 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff4fa80-fa76-424e-93e9-0fd553ec34f3 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:51.289392 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff4fa80-fa76-424e-93e9-0fd553ec34f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:51.319106 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-fff4fa80-fa76-424e-93e9-0fd553ec34f3 None None] MFA Rules not processed for user `bd3967ff06eb4f18bdebd396dbfa2d65`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:51.320490 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-fff4fa80-fa76-424e-93e9-0fd553ec34f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:51.429890 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 835/1674] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:51 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:51.493533 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2ef3e6-d16e-4f44-9415-a614a55582c3 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:51.495444 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-4f2ef3e6-d16e-4f44-9415-a614a55582c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:51.568636 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2ef3e6-d16e-4f44-9415-a614a55582c3 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:51.574035 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04d2d53-99b7-4e72-8916-07138ec52165 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:51.574711 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04d2d53-99b7-4e72-8916-07138ec52165 service cinder] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:51.574985 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04d2d53-99b7-4e72-8916-07138ec52165 service cinder] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:51.579841 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04d2d53-99b7-4e72-8916-07138ec52165 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:51.579841 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-e04d2d53-99b7-4e72-8916-07138ec52165 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:51.650556 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04d2d53-99b7-4e72-8916-07138ec52165 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:51.652063 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04d2d53-99b7-4e72-8916-07138ec52165 service cinder] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:51.677378 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 840/1675] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:20:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:20:51.753813 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:52.030978 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:52.329452 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff4fa80-fa76-424e-93e9-0fd553ec34f3 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:52.330898 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-fff4fa80-fa76-424e-93e9-0fd553ec34f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:52.435432 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff4fa80-fa76-424e-93e9-0fd553ec34f3 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:52.440261 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1507627-8c98-4058-ad11-fc9c24420c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:52.440924 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1507627-8c98-4058-ad11-fc9c24420c98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:52.441205 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1507627-8c98-4058-ad11-fc9c24420c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:52.441481 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1507627-8c98-4058-ad11-fc9c24420c98 admin admin] PATH_INFO: `/v3/users/4f637440d2a04fa2a5bf6eccf5e5a4a3` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:52.442670 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1507627-8c98-4058-ad11-fc9c24420c98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f637440d2a04fa2a5bf6eccf5e5a4a3)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:52.444672 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1507627-8c98-4058-ad11-fc9c24420c98 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:52.621571 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b6299e-9c78-47c6-a78a-9d2f2d824043 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:52.621571 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b6299e-9c78-47c6-a78a-9d2f2d824043 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:52.621571 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b6299e-9c78-47c6-a78a-9d2f2d824043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:52.653458 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-84b6299e-9c78-47c6-a78a-9d2f2d824043 None None] MFA Rules not processed for user `a491001660e549db85a151e77711797c`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:52.654786 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-84b6299e-9c78-47c6-a78a-9d2f2d824043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:52.719805 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-d1507627-8c98-4058-ad11-fc9c24420c98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f637440d2a04fa2a5bf6eccf5e5a4a3'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:52.720485 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-d1507627-8c98-4058-ad11-fc9c24420c98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f637440d2a04fa2a5bf6eccf5e5a4a3'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:52.734113 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-d1507627-8c98-4058-ad11-fc9c24420c98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f637440d2a04fa2a5bf6eccf5e5a4a3'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:52.741148 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 841/1676] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 00:20:52.742607 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b6299e-9c78-47c6-a78a-9d2f2d824043 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:52.744402 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-84b6299e-9c78-47c6-a78a-9d2f2d824043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:52.758947 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b6299e-9c78-47c6-a78a-9d2f2d824043 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:52.768689 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade9ce03-dfd1-41f3-ba3c-d3c37fb52bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:52.769312 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade9ce03-dfd1-41f3-ba3c-d3c37fb52bef admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:52.769591 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade9ce03-dfd1-41f3-ba3c-d3c37fb52bef admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:52.769926 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade9ce03-dfd1-41f3-ba3c-d3c37fb52bef admin admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:52.770562 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade9ce03-dfd1-41f3-ba3c-d3c37fb52bef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:52.772805 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade9ce03-dfd1-41f3-ba3c-d3c37fb52bef admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:52.825186 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-d1507627-8c98-4058-ad11-fc9c24420c98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f637440d2a04fa2a5bf6eccf5e5a4a3'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:52.825809 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-d1507627-8c98-4058-ad11-fc9c24420c98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f637440d2a04fa2a5bf6eccf5e5a4a3'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:52.838882 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 836/1677] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:20:52 2022] DELETE /identity/v3/users/4f637440d2a04fa2a5bf6eccf5e5a4a3 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:52.843310 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e82f1ed-38f7-4aad-9692-5a7e707f8b02 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:52.843705 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e82f1ed-38f7-4aad-9692-5a7e707f8b02 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:52.844055 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e82f1ed-38f7-4aad-9692-5a7e707f8b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:52.862661 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 842/1678] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:20:52 2022] POST /identity/v3/projects => generated 373 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:20:52.863978 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade9ce03-dfd1-41f3-ba3c-d3c37fb52bef admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:52.866042 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-2e82f1ed-38f7-4aad-9692-5a7e707f8b02 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:52.867415 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-2e82f1ed-38f7-4aad-9692-5a7e707f8b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:52.933012 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 837/1679] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 00:20:52.935480 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade9ce03-dfd1-41f3-ba3c-d3c37fb52bef admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:52.942540 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e82f1ed-38f7-4aad-9692-5a7e707f8b02 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:52.943918 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-2e82f1ed-38f7-4aad-9692-5a7e707f8b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:52.944943 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aa524e-0fd7-40ec-bf60-3b4d607d72d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:52.945569 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aa524e-0fd7-40ec-bf60-3b4d607d72d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:52.945854 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aa524e-0fd7-40ec-bf60-3b4d607d72d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:52.946271 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aa524e-0fd7-40ec-bf60-3b4d607d72d3 admin admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:52.946916 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aa524e-0fd7-40ec-bf60-3b4d607d72d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:52.949003 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aa524e-0fd7-40ec-bf60-3b4d607d72d3 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:53.018480 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e82f1ed-38f7-4aad-9692-5a7e707f8b02 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:53.023004 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946ed290-77fa-4eb9-b87c-bf5644cc492e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:53.023842 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946ed290-77fa-4eb9-b87c-bf5644cc492e service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:53.024211 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946ed290-77fa-4eb9-b87c-bf5644cc492e service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:53.024615 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946ed290-77fa-4eb9-b87c-bf5644cc492e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:53.040995 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 843/1680] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:20:52 2022] POST /identity/v3/users => generated 293 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:20:53.045670 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aa524e-0fd7-40ec-bf60-3b4d607d72d3 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:53.053013 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946ed290-77fa-4eb9-b87c-bf5644cc492e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:53.056413 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946ed290-77fa-4eb9-b87c-bf5644cc492e service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:53.062096 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aa524e-0fd7-40ec-bf60-3b4d607d72d3 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:53.071000 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a6d1db-48cc-460f-a003-55d8f51ba3b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:53.072391 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a6d1db-48cc-460f-a003-55d8f51ba3b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:53.072734 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a6d1db-48cc-460f-a003-55d8f51ba3b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:53.073320 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a6d1db-48cc-460f-a003-55d8f51ba3b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:53.075886 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a6d1db-48cc-460f-a003-55d8f51ba3b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:53.077452 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 838/1681] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:20:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 00:20:53.078706 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a6d1db-48cc-460f-a003-55d8f51ba3b0 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:53.090112 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 844/1682] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:20:53 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 00:20:53.094494 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946ed290-77fa-4eb9-b87c-bf5644cc492e service neutron] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:53.105251 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946ed290-77fa-4eb9-b87c-bf5644cc492e service neutron] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:53.115805 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf70905-6b39-4513-af7a-d6fa46b28c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:53.115805 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf70905-6b39-4513-af7a-d6fa46b28c89 admin admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:53.115805 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf70905-6b39-4513-af7a-d6fa46b28c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:53.115805 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf70905-6b39-4513-af7a-d6fa46b28c89 admin admin] PATH_INFO: `/v3/projects/c809115d8a164a3992177fcd20ebc96a/users/a2d0b932ba36433bb22f5eb15ad2c180/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:53.143888 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf70905-6b39-4513-af7a-d6fa46b28c89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c809115d8a164a3992177fcd20ebc96a, user_id=a2d0b932ba36433bb22f5eb15ad2c180, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:53.147052 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf70905-6b39-4513-af7a-d6fa46b28c89 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:53.237737 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 839/1683] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:20:53 2022] PUT /identity/v3/projects/c809115d8a164a3992177fcd20ebc96a/users/a2d0b932ba36433bb22f5eb15ad2c180/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:53.241973 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a6d1db-48cc-460f-a003-55d8f51ba3b0 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:53.260118 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a6d1db-48cc-460f-a003-55d8f51ba3b0 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:53.347316 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa06992-89b7-4d08-ae86-e8cc8cf45691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:53.348839 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa06992-89b7-4d08-ae86-e8cc8cf45691 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:53.348839 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa06992-89b7-4d08-ae86-e8cc8cf45691 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:53.348839 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa06992-89b7-4d08-ae86-e8cc8cf45691 admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:53.349218 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa06992-89b7-4d08-ae86-e8cc8cf45691 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:53.350826 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa06992-89b7-4d08-ae86-e8cc8cf45691 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:53.362848 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 845/1684] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:20:53 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:20:53.367199 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf70905-6b39-4513-af7a-d6fa46b28c89 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:53.401256 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa06992-89b7-4d08-ae86-e8cc8cf45691 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:53.408422 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf70905-6b39-4513-af7a-d6fa46b28c89 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:53.414041 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-ffa06992-89b7-4d08-ae86-e8cc8cf45691 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:53.420919 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98d3629-b5fe-494e-80d5-299579237881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:53.421622 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98d3629-b5fe-494e-80d5-299579237881 admin admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:53.423082 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98d3629-b5fe-494e-80d5-299579237881 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:53.423406 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98d3629-b5fe-494e-80d5-299579237881 admin admin] PATH_INFO: `/v3/projects/c809115d8a164a3992177fcd20ebc96a/users/a2d0b932ba36433bb22f5eb15ad2c180/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:53.427779 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98d3629-b5fe-494e-80d5-299579237881 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c809115d8a164a3992177fcd20ebc96a, user_id=a2d0b932ba36433bb22f5eb15ad2c180, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:53.442007 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98d3629-b5fe-494e-80d5-299579237881 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:53.442717 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa06992-89b7-4d08-ae86-e8cc8cf45691 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:53.450609 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652ed1cd-daca-4bc3-8eb9-07f75d7ef734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:53.451728 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652ed1cd-daca-4bc3-8eb9-07f75d7ef734 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:53.452003 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652ed1cd-daca-4bc3-8eb9-07f75d7ef734 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:53.452556 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652ed1cd-daca-4bc3-8eb9-07f75d7ef734 admin admin] PATH_INFO: `/v3/users/4cf349d7b4bf48a580976dc4e0ab8d48` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:53.453707 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652ed1cd-daca-4bc3-8eb9-07f75d7ef734 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cf349d7b4bf48a580976dc4e0ab8d48)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:53.457165 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652ed1cd-daca-4bc3-8eb9-07f75d7ef734 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:53.479797 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 840/1685] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:20:53 2022] PUT /identity/v3/projects/c809115d8a164a3992177fcd20ebc96a/users/a2d0b932ba36433bb22f5eb15ad2c180/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 00:20:53.733662 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-652ed1cd-daca-4bc3-8eb9-07f75d7ef734 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cf349d7b4bf48a580976dc4e0ab8d48'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:53.735880 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-652ed1cd-daca-4bc3-8eb9-07f75d7ef734 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cf349d7b4bf48a580976dc4e0ab8d48'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:53.757603 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:53.779534 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-652ed1cd-daca-4bc3-8eb9-07f75d7ef734 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cf349d7b4bf48a580976dc4e0ab8d48'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:53.831025 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:53.857276 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98d3629-b5fe-494e-80d5-299579237881 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:53.859056 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-b98d3629-b5fe-494e-80d5-299579237881 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:53.859716 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-652ed1cd-daca-4bc3-8eb9-07f75d7ef734 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cf349d7b4bf48a580976dc4e0ab8d48'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:53.860682 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-652ed1cd-daca-4bc3-8eb9-07f75d7ef734 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cf349d7b4bf48a580976dc4e0ab8d48'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:53.876124 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 846/1686] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:20:53 2022] DELETE /identity/v3/users/4cf349d7b4bf48a580976dc4e0ab8d48 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:53.884217 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e4f984-a4b4-4cfa-b7ef-a61595426abe None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:53.884857 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e4f984-a4b4-4cfa-b7ef-a61595426abe None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:53.886194 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e4f984-a4b4-4cfa-b7ef-a61595426abe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:53.922049 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-d1e4f984-a4b4-4cfa-b7ef-a61595426abe None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:53.923480 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-d1e4f984-a4b4-4cfa-b7ef-a61595426abe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:53.984363 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98d3629-b5fe-494e-80d5-299579237881 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:54.108181 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd7291f-8705-43fd-9662-27c3b39ca262 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:54.108797 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd7291f-8705-43fd-9662-27c3b39ca262 service cinder] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:54.109081 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd7291f-8705-43fd-9662-27c3b39ca262 service cinder] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:54.109353 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd7291f-8705-43fd-9662-27c3b39ca262 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:54.111844 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-1bd7291f-8705-43fd-9662-27c3b39ca262 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:54.118694 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 847/1687] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 00:20:54.139946 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e4f984-a4b4-4cfa-b7ef-a61595426abe None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:54.141781 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-d1e4f984-a4b4-4cfa-b7ef-a61595426abe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:54.220582 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e4f984-a4b4-4cfa-b7ef-a61595426abe None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:54.226941 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e7c541-5432-4cbf-8361-5a77ce9ac4ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:54.226941 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e7c541-5432-4cbf-8361-5a77ce9ac4ab service neutron] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:54.227230 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e7c541-5432-4cbf-8361-5a77ce9ac4ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:54.227452 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e7c541-5432-4cbf-8361-5a77ce9ac4ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:54.244302 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e7c541-5432-4cbf-8361-5a77ce9ac4ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:54.247305 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e7c541-5432-4cbf-8361-5a77ce9ac4ab service neutron] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:54.252635 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd7291f-8705-43fd-9662-27c3b39ca262 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:54.253794 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd7291f-8705-43fd-9662-27c3b39ca262 service cinder] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:54.272916 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 848/1688] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:20:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:20:54.284816 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 841/1689] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:20:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 00:20:54.379251 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e7c541-5432-4cbf-8361-5a77ce9ac4ab service neutron] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:54.381072 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-64e7c541-5432-4cbf-8361-5a77ce9ac4ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:54.403735 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e7c541-5432-4cbf-8361-5a77ce9ac4ab service neutron] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:54.416069 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c665f9f-9579-48b4-9a8a-1fffc79cd5bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:54.416879 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c665f9f-9579-48b4-9a8a-1fffc79cd5bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:54.417162 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c665f9f-9579-48b4-9a8a-1fffc79cd5bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:54.417435 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c665f9f-9579-48b4-9a8a-1fffc79cd5bb admin admin] PATH_INFO: `/v3/projects/4482691531e044679104fbaba77c30d3` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:54.420053 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c665f9f-9579-48b4-9a8a-1fffc79cd5bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=4482691531e044679104fbaba77c30d3)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:54.423237 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c665f9f-9579-48b4-9a8a-1fffc79cd5bb admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:54.618810 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-7c665f9f-9579-48b4-9a8a-1fffc79cd5bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4482691531e044679104fbaba77c30d3', 'request_id': 'req-7c665f9f-9579-48b4-9a8a-1fffc79cd5bb'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:54.630162 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-7c665f9f-9579-48b4-9a8a-1fffc79cd5bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4482691531e044679104fbaba77c30d3', 'request_id': 'req-7c665f9f-9579-48b4-9a8a-1fffc79cd5bb'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:54.770890 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-7c665f9f-9579-48b4-9a8a-1fffc79cd5bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4482691531e044679104fbaba77c30d3', 'request_id': 'req-7c665f9f-9579-48b4-9a8a-1fffc79cd5bb'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:54.797778 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-7c665f9f-9579-48b4-9a8a-1fffc79cd5bb admin admin] The token cache is being invalidate because project 4482691531e044679104fbaba77c30d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81747) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:20:54.798252 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-7c665f9f-9579-48b4-9a8a-1fffc79cd5bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:54.799083 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-7c665f9f-9579-48b4-9a8a-1fffc79cd5bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:54.801119 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 849/1690] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:20:54 2022] DELETE /identity/v3/projects/4482691531e044679104fbaba77c30d3 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:54.804782 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd7291f-8705-43fd-9662-27c3b39ca262 service cinder] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:54.835134 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd7291f-8705-43fd-9662-27c3b39ca262 service cinder] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:54.914013 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e84bac-c907-4441-89a6-ddbf593e10f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:54.914739 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e84bac-c907-4441-89a6-ddbf593e10f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:54.915023 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e84bac-c907-4441-89a6-ddbf593e10f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:54.915299 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e84bac-c907-4441-89a6-ddbf593e10f0 admin admin] PATH_INFO: `/v3/users/15969b140423409b89e7491f0fb191e4` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:54.917509 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e84bac-c907-4441-89a6-ddbf593e10f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15969b140423409b89e7491f0fb191e4)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:54.920028 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e84bac-c907-4441-89a6-ddbf593e10f0 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:54.934082 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:55.222158 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-27e84bac-c907-4441-89a6-ddbf593e10f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15969b140423409b89e7491f0fb191e4'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:55.223216 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-27e84bac-c907-4441-89a6-ddbf593e10f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15969b140423409b89e7491f0fb191e4'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:55.246694 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-27e84bac-c907-4441-89a6-ddbf593e10f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15969b140423409b89e7491f0fb191e4'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:55.334206 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-27e84bac-c907-4441-89a6-ddbf593e10f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15969b140423409b89e7491f0fb191e4'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:55.336366 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-27e84bac-c907-4441-89a6-ddbf593e10f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15969b140423409b89e7491f0fb191e4'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:55.348964 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c665f9f-9579-48b4-9a8a-1fffc79cd5bb admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:55.351826 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-7c665f9f-9579-48b4-9a8a-1fffc79cd5bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:55.368683 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 842/1691] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:20:54 2022] DELETE /identity/v3/users/15969b140423409b89e7491f0fb191e4 => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:55.380706 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c665f9f-9579-48b4-9a8a-1fffc79cd5bb admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:55.466478 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd07a0b5-bba5-4d30-a9e1-d6386f123fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:55.467118 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd07a0b5-bba5-4d30-a9e1-d6386f123fd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:55.467587 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd07a0b5-bba5-4d30-a9e1-d6386f123fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:55.467875 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd07a0b5-bba5-4d30-a9e1-d6386f123fd9 admin admin] PATH_INFO: `/v3/projects/dff76c0c26d34f41835465f2d900605f` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:55.469033 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd07a0b5-bba5-4d30-a9e1-d6386f123fd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dff76c0c26d34f41835465f2d900605f)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:55.470958 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd07a0b5-bba5-4d30-a9e1-d6386f123fd9 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:55.652931 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-fd07a0b5-bba5-4d30-a9e1-d6386f123fd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dff76c0c26d34f41835465f2d900605f', 'request_id': 'req-fd07a0b5-bba5-4d30-a9e1-d6386f123fd9'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:55.659148 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-fd07a0b5-bba5-4d30-a9e1-d6386f123fd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dff76c0c26d34f41835465f2d900605f', 'request_id': 'req-fd07a0b5-bba5-4d30-a9e1-d6386f123fd9'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:55.713861 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:55.756059 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-fd07a0b5-bba5-4d30-a9e1-d6386f123fd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dff76c0c26d34f41835465f2d900605f', 'request_id': 'req-fd07a0b5-bba5-4d30-a9e1-d6386f123fd9'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:55.771869 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-fd07a0b5-bba5-4d30-a9e1-d6386f123fd9 admin admin] The token cache is being invalidate because project dff76c0c26d34f41835465f2d900605f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81747) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:20:55.773079 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-fd07a0b5-bba5-4d30-a9e1-d6386f123fd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:55.774595 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-fd07a0b5-bba5-4d30-a9e1-d6386f123fd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:55.776590 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 850/1692] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:20:55 2022] DELETE /identity/v3/projects/dff76c0c26d34f41835465f2d900605f => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:55.781367 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e84bac-c907-4441-89a6-ddbf593e10f0 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:55.782994 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-27e84bac-c907-4441-89a6-ddbf593e10f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:55.796425 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e84bac-c907-4441-89a6-ddbf593e10f0 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:55.875287 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7bc9f8-bdf6-4e3f-a990-c5d04a834461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:55.877384 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7bc9f8-bdf6-4e3f-a990-c5d04a834461 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:55.877830 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7bc9f8-bdf6-4e3f-a990-c5d04a834461 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:55.878296 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7bc9f8-bdf6-4e3f-a990-c5d04a834461 admin admin] PATH_INFO: `/v3/users/45453d4147d949e1a6cff860a0f86fa1` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:55.880371 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7bc9f8-bdf6-4e3f-a990-c5d04a834461 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45453d4147d949e1a6cff860a0f86fa1)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:55.888659 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7bc9f8-bdf6-4e3f-a990-c5d04a834461 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:56.046232 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd07a0b5-bba5-4d30-a9e1-d6386f123fd9 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:56.047762 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-fd07a0b5-bba5-4d30-a9e1-d6386f123fd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:56.114176 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd07a0b5-bba5-4d30-a9e1-d6386f123fd9 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:56.123947 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccf39d6-3c74-482a-a502-350c78ecb798 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:56.125955 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccf39d6-3c74-482a-a502-350c78ecb798 service cinder] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:56.126240 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccf39d6-3c74-482a-a502-350c78ecb798 service cinder] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:56.126509 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccf39d6-3c74-482a-a502-350c78ecb798 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:56.130980 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-fccf39d6-3c74-482a-a502-350c78ecb798 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:56.151757 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:56.156559 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-cb7bc9f8-bdf6-4e3f-a990-c5d04a834461 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45453d4147d949e1a6cff860a0f86fa1'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:56.158200 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-cb7bc9f8-bdf6-4e3f-a990-c5d04a834461 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45453d4147d949e1a6cff860a0f86fa1'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:56.190470 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-cb7bc9f8-bdf6-4e3f-a990-c5d04a834461 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45453d4147d949e1a6cff860a0f86fa1'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:56.222047 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:56.274880 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-cb7bc9f8-bdf6-4e3f-a990-c5d04a834461 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45453d4147d949e1a6cff860a0f86fa1'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:56.275804 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-cb7bc9f8-bdf6-4e3f-a990-c5d04a834461 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45453d4147d949e1a6cff860a0f86fa1'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:56.283896 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccf39d6-3c74-482a-a502-350c78ecb798 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:56.289689 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccf39d6-3c74-482a-a502-350c78ecb798 service cinder] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:56.302333 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 843/1693] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:20:55 2022] DELETE /identity/v3/users/45453d4147d949e1a6cff860a0f86fa1 => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:56.302333 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ba659-6e79-4277-8b97-6b3f64c0c457 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:56.302333 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ba659-6e79-4277-8b97-6b3f64c0c457 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:56.305980 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ba659-6e79-4277-8b97-6b3f64c0c457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:56.334274 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 851/1694] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:20:56 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 00:20:56.368052 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-e12ba659-6e79-4277-8b97-6b3f64c0c457 None None] MFA Rules not processed for user `a2d0b932ba36433bb22f5eb15ad2c180`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:56.369948 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-e12ba659-6e79-4277-8b97-6b3f64c0c457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:56.449843 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccf39d6-3c74-482a-a502-350c78ecb798 service cinder] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:56.451956 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-fccf39d6-3c74-482a-a502-350c78ecb798 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:56.474229 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccf39d6-3c74-482a-a502-350c78ecb798 service cinder] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:56.497751 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 844/1695] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:56 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:56.514304 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12ba659-6e79-4277-8b97-6b3f64c0c457 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:56.524680 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-e12ba659-6e79-4277-8b97-6b3f64c0c457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:56.591561 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a967aaa-9eff-45b2-8f5d-12126d1e4e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:56.592227 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a967aaa-9eff-45b2-8f5d-12126d1e4e65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:56.592497 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a967aaa-9eff-45b2-8f5d-12126d1e4e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:56.592775 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a967aaa-9eff-45b2-8f5d-12126d1e4e65 admin admin] PATH_INFO: `/v3/projects/296f3fd4378b4da2a020039bd67613a3` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:56.594001 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a967aaa-9eff-45b2-8f5d-12126d1e4e65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=296f3fd4378b4da2a020039bd67613a3)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:56.596086 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a967aaa-9eff-45b2-8f5d-12126d1e4e65 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:56.637082 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12ba659-6e79-4277-8b97-6b3f64c0c457 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:56.643296 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f8d06b-6380-4dd3-891d-a518173e09fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:56.645318 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f8d06b-6380-4dd3-891d-a518173e09fc service cinder] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:56.646473 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f8d06b-6380-4dd3-891d-a518173e09fc service cinder] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:56.646849 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f8d06b-6380-4dd3-891d-a518173e09fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:56.656120 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-03f8d06b-6380-4dd3-891d-a518173e09fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:56.743964 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f8d06b-6380-4dd3-891d-a518173e09fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:56.745914 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f8d06b-6380-4dd3-891d-a518173e09fc service cinder] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:56.773816 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 845/1696] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:20:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:20:56.800948 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-7a967aaa-9eff-45b2-8f5d-12126d1e4e65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '296f3fd4378b4da2a020039bd67613a3', 'request_id': 'req-7a967aaa-9eff-45b2-8f5d-12126d1e4e65'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:56.813066 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-7a967aaa-9eff-45b2-8f5d-12126d1e4e65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '296f3fd4378b4da2a020039bd67613a3', 'request_id': 'req-7a967aaa-9eff-45b2-8f5d-12126d1e4e65'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:56.935751 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-7a967aaa-9eff-45b2-8f5d-12126d1e4e65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '296f3fd4378b4da2a020039bd67613a3', 'request_id': 'req-7a967aaa-9eff-45b2-8f5d-12126d1e4e65'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:56.946826 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-7a967aaa-9eff-45b2-8f5d-12126d1e4e65 admin admin] The token cache is being invalidate because project 296f3fd4378b4da2a020039bd67613a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81747) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:20:56.947113 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-7a967aaa-9eff-45b2-8f5d-12126d1e4e65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:56.947966 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-7a967aaa-9eff-45b2-8f5d-12126d1e4e65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:56.950002 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 852/1697] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:20:56 2022] DELETE /identity/v3/projects/296f3fd4378b4da2a020039bd67613a3 => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 00:20:56.954217 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f8d06b-6380-4dd3-891d-a518173e09fc service cinder] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:56.955687 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-03f8d06b-6380-4dd3-891d-a518173e09fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:56.975727 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f8d06b-6380-4dd3-891d-a518173e09fc service cinder] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:56.982602 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a967aaa-9eff-45b2-8f5d-12126d1e4e65 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:56.987574 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-7a967aaa-9eff-45b2-8f5d-12126d1e4e65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:57.088268 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a967aaa-9eff-45b2-8f5d-12126d1e4e65 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:57.093197 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285139e3-2a06-47ff-b92d-55dd0f7834bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:57.094228 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285139e3-2a06-47ff-b92d-55dd0f7834bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:57.094392 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2b484f-7040-4cbf-8637-254ec4751f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:57.094811 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285139e3-2a06-47ff-b92d-55dd0f7834bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:57.095154 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b484f-7040-4cbf-8637-254ec4751f33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:57.095497 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285139e3-2a06-47ff-b92d-55dd0f7834bd admin admin] PATH_INFO: `/v3/users/f5b81caad7c5408ab79d800ad57f0c4e` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:57.095608 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b484f-7040-4cbf-8637-254ec4751f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:57.095827 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b484f-7040-4cbf-8637-254ec4751f33 admin admin] PATH_INFO: `/v3/projects/5e8a6efec6224399b60321ae8367a9d2` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:57.097081 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285139e3-2a06-47ff-b92d-55dd0f7834bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5b81caad7c5408ab79d800ad57f0c4e)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:57.097198 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2b484f-7040-4cbf-8637-254ec4751f33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e8a6efec6224399b60321ae8367a9d2)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:57.099537 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2b484f-7040-4cbf-8637-254ec4751f33 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:57.099812 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285139e3-2a06-47ff-b92d-55dd0f7834bd admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:57.311625 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-285139e3-2a06-47ff-b92d-55dd0f7834bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5b81caad7c5408ab79d800ad57f0c4e'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:57.312994 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-285139e3-2a06-47ff-b92d-55dd0f7834bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5b81caad7c5408ab79d800ad57f0c4e'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:57.314544 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-ac2b484f-7040-4cbf-8637-254ec4751f33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e8a6efec6224399b60321ae8367a9d2', 'request_id': 'req-ac2b484f-7040-4cbf-8637-254ec4751f33'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:57.323329 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-ac2b484f-7040-4cbf-8637-254ec4751f33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e8a6efec6224399b60321ae8367a9d2', 'request_id': 'req-ac2b484f-7040-4cbf-8637-254ec4751f33'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:57.327800 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-285139e3-2a06-47ff-b92d-55dd0f7834bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5b81caad7c5408ab79d800ad57f0c4e'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:57.407220 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-ac2b484f-7040-4cbf-8637-254ec4751f33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e8a6efec6224399b60321ae8367a9d2', 'request_id': 'req-ac2b484f-7040-4cbf-8637-254ec4751f33'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:57.422470 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-ac2b484f-7040-4cbf-8637-254ec4751f33 admin admin] The token cache is being invalidate because project 5e8a6efec6224399b60321ae8367a9d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81747) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:20:57.422470 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-ac2b484f-7040-4cbf-8637-254ec4751f33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:57.422470 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-ac2b484f-7040-4cbf-8637-254ec4751f33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:57.423832 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 853/1698] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:20:56 2022] DELETE /identity/v3/projects/5e8a6efec6224399b60321ae8367a9d2 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 00:20:57.429818 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf85bb8-f4f6-464c-865b-f6696f2109fd None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:57.430102 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf85bb8-f4f6-464c-865b-f6696f2109fd None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:57.430378 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf85bb8-f4f6-464c-865b-f6696f2109fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:57.451875 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-4cf85bb8-f4f6-464c-865b-f6696f2109fd None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:57.453097 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-4cf85bb8-f4f6-464c-865b-f6696f2109fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:57.481775 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-285139e3-2a06-47ff-b92d-55dd0f7834bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5b81caad7c5408ab79d800ad57f0c4e'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:57.482405 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-285139e3-2a06-47ff-b92d-55dd0f7834bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5b81caad7c5408ab79d800ad57f0c4e'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:57.496680 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 846/1699] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:20:56 2022] DELETE /identity/v3/users/f5b81caad7c5408ab79d800ad57f0c4e => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:57.521378 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 854/1700] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 00:20:57.529406 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01790dfc-ed7a-46ca-b0c8-f383407f04c0 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:57.529764 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01790dfc-ed7a-46ca-b0c8-f383407f04c0 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:57.530047 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01790dfc-ed7a-46ca-b0c8-f383407f04c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:57.568756 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-01790dfc-ed7a-46ca-b0c8-f383407f04c0 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:57.570064 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-01790dfc-ed7a-46ca-b0c8-f383407f04c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:57.647742 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 847/1701] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:57.652051 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf85bb8-f4f6-464c-865b-f6696f2109fd None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:57.662315 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf85bb8-f4f6-464c-865b-f6696f2109fd None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:57.667132 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68410e73-845b-4b00-843c-7479966a2408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:57.668357 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68410e73-845b-4b00-843c-7479966a2408 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:57.668573 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68410e73-845b-4b00-843c-7479966a2408 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:57.668838 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68410e73-845b-4b00-843c-7479966a2408 admin admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:57.670487 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68410e73-845b-4b00-843c-7479966a2408 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:20:57.670487 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68410e73-845b-4b00-843c-7479966a2408 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:57.671408 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68410e73-845b-4b00-843c-7479966a2408 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:57.689272 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 855/1702] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Oct 12 00:20:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:20:57.693818 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01790dfc-ed7a-46ca-b0c8-f383407f04c0 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:57.705352 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01790dfc-ed7a-46ca-b0c8-f383407f04c0 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:57.707454 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581a5157-073d-4773-8a4a-4138dfdae2f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:57.708199 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581a5157-073d-4773-8a4a-4138dfdae2f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:57.708542 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581a5157-073d-4773-8a4a-4138dfdae2f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:57.708878 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581a5157-073d-4773-8a4a-4138dfdae2f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:57.709834 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581a5157-073d-4773-8a4a-4138dfdae2f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:57.712193 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581a5157-073d-4773-8a4a-4138dfdae2f4 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:57.838228 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 848/1703] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:20:57 2022] POST /identity/v3/projects => generated 357 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:20:57.839460 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68410e73-845b-4b00-843c-7479966a2408 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:57.850499 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68410e73-845b-4b00-843c-7479966a2408 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:57.911001 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732d29e7-ae68-41c3-a1ba-275541ced98a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:57.911652 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732d29e7-ae68-41c3-a1ba-275541ced98a admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:57.911927 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732d29e7-ae68-41c3-a1ba-275541ced98a admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:57.912192 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732d29e7-ae68-41c3-a1ba-275541ced98a admin admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:57.912817 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732d29e7-ae68-41c3-a1ba-275541ced98a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:57.914919 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732d29e7-ae68-41c3-a1ba-275541ced98a admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:57.953485 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:58.022899 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581a5157-073d-4773-8a4a-4138dfdae2f4 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:58.022899 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-581a5157-073d-4773-8a4a-4138dfdae2f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:58.036367 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581a5157-073d-4773-8a4a-4138dfdae2f4 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:58.041361 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcb5ad0-a5e3-485e-974e-8a7a3fd4a3cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:58.042002 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcb5ad0-a5e3-485e-974e-8a7a3fd4a3cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:58.042281 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcb5ad0-a5e3-485e-974e-8a7a3fd4a3cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:58.042556 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcb5ad0-a5e3-485e-974e-8a7a3fd4a3cb admin admin] PATH_INFO: `/v3/projects/99208e8c337f4206a93a1ef96a3c13da` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:58.044297 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcb5ad0-a5e3-485e-974e-8a7a3fd4a3cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=99208e8c337f4206a93a1ef96a3c13da)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:58.046256 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcb5ad0-a5e3-485e-974e-8a7a3fd4a3cb admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:58.069636 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:58.075036 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 856/1704] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:20:57 2022] POST /identity/v3/users => generated 279 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:20:58.080468 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732d29e7-ae68-41c3-a1ba-275541ced98a admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:58.091070 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732d29e7-ae68-41c3-a1ba-275541ced98a admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:58.095829 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9d474d-95a6-4cfc-9b9e-4bff1d46b5f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:58.096423 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9d474d-95a6-4cfc-9b9e-4bff1d46b5f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:58.096747 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9d474d-95a6-4cfc-9b9e-4bff1d46b5f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:58.097015 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9d474d-95a6-4cfc-9b9e-4bff1d46b5f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:58.097668 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9d474d-95a6-4cfc-9b9e-4bff1d46b5f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:58.099242 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9d474d-95a6-4cfc-9b9e-4bff1d46b5f5 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:58.110856 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 857/1705] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:20:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:20:58.114751 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9d474d-95a6-4cfc-9b9e-4bff1d46b5f5 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:58.124629 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9d474d-95a6-4cfc-9b9e-4bff1d46b5f5 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:58.129295 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa37e4e4-5f4a-42aa-9de0-5d60728fd001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:58.129878 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa37e4e4-5f4a-42aa-9de0-5d60728fd001 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:58.130146 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa37e4e4-5f4a-42aa-9de0-5d60728fd001 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:58.130436 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa37e4e4-5f4a-42aa-9de0-5d60728fd001 admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:58.131143 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa37e4e4-5f4a-42aa-9de0-5d60728fd001 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:58.132743 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa37e4e4-5f4a-42aa-9de0-5d60728fd001 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:58.142594 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 858/1706] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:20:58 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:20:58.146443 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa37e4e4-5f4a-42aa-9de0-5d60728fd001 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:58.156335 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa37e4e4-5f4a-42aa-9de0-5d60728fd001 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:58.160767 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd73ac4-0ccc-41ad-a1f5-d882a26c604e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:58.161367 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd73ac4-0ccc-41ad-a1f5-d882a26c604e admin admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:58.161637 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd73ac4-0ccc-41ad-a1f5-d882a26c604e admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:58.161900 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd73ac4-0ccc-41ad-a1f5-d882a26c604e admin admin] PATH_INFO: `/v3/projects/b5ce5a2e842041f3ab5a22f8fa519960/users/b84f16d32146489fb6dcab1b29c5bda9/roles/ef689f96442945e8a7bc8cbfee13d909` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:58.178016 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd73ac4-0ccc-41ad-a1f5-d882a26c604e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5ce5a2e842041f3ab5a22f8fa519960, user_id=b84f16d32146489fb6dcab1b29c5bda9, role_id=ef689f96442945e8a7bc8cbfee13d909)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:58.180904 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd73ac4-0ccc-41ad-a1f5-d882a26c604e admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:58.191241 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-5bcb5ad0-a5e3-485e-974e-8a7a3fd4a3cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99208e8c337f4206a93a1ef96a3c13da', 'request_id': 'req-5bcb5ad0-a5e3-485e-974e-8a7a3fd4a3cb'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:58.197560 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-5bcb5ad0-a5e3-485e-974e-8a7a3fd4a3cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99208e8c337f4206a93a1ef96a3c13da', 'request_id': 'req-5bcb5ad0-a5e3-485e-974e-8a7a3fd4a3cb'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:58.272307 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-5bcb5ad0-a5e3-485e-974e-8a7a3fd4a3cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99208e8c337f4206a93a1ef96a3c13da', 'request_id': 'req-5bcb5ad0-a5e3-485e-974e-8a7a3fd4a3cb'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:58.282908 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 859/1707] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:20:58 2022] PUT /identity/v3/projects/b5ce5a2e842041f3ab5a22f8fa519960/users/b84f16d32146489fb6dcab1b29c5bda9/roles/ef689f96442945e8a7bc8cbfee13d909 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 00:20:58.295759 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da5bef2-9c78-4ff9-af59-6cd970d2757b None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:58.296147 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da5bef2-9c78-4ff9-af59-6cd970d2757b None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:58.296259 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-5bcb5ad0-a5e3-485e-974e-8a7a3fd4a3cb admin admin] The token cache is being invalidate because project 99208e8c337f4206a93a1ef96a3c13da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81746) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:20:58.298078 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-5bcb5ad0-a5e3-485e-974e-8a7a3fd4a3cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:58.298316 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da5bef2-9c78-4ff9-af59-6cd970d2757b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:58.298406 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-5bcb5ad0-a5e3-485e-974e-8a7a3fd4a3cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:58.299661 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 849/1708] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:20:58 2022] DELETE /identity/v3/projects/99208e8c337f4206a93a1ef96a3c13da => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 00:20:58.305273 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdfa6a1-8e60-4d6b-85e7-586243a320ed None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:58.306261 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdfa6a1-8e60-4d6b-85e7-586243a320ed None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:58.306691 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdfa6a1-8e60-4d6b-85e7-586243a320ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:58.335277 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-8da5bef2-9c78-4ff9-af59-6cd970d2757b None None] MFA Rules not processed for user `b84f16d32146489fb6dcab1b29c5bda9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:58.337241 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-8da5bef2-9c78-4ff9-af59-6cd970d2757b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:58.356436 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-2bdfa6a1-8e60-4d6b-85e7-586243a320ed None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:58.357748 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-2bdfa6a1-8e60-4d6b-85e7-586243a320ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:58.426370 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 860/1709] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:58 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:58.443069 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 850/1710] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:58.451885 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ec8f3a-c2c8-4803-8d6d-ebef59a55c79 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:58.452169 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ec8f3a-c2c8-4803-8d6d-ebef59a55c79 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:58.452438 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ec8f3a-c2c8-4803-8d6d-ebef59a55c79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:58.472616 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdfa6a1-8e60-4d6b-85e7-586243a320ed None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:58.474009 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-2bdfa6a1-8e60-4d6b-85e7-586243a320ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:58.479032 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-54ec8f3a-c2c8-4803-8d6d-ebef59a55c79 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:58.480329 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-54ec8f3a-c2c8-4803-8d6d-ebef59a55c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:58.500192 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 861/1711] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:58.501461 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ec8f3a-c2c8-4803-8d6d-ebef59a55c79 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:58.510580 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ec8f3a-c2c8-4803-8d6d-ebef59a55c79 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:58.516782 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4723a70-acd3-431d-9076-4e9dfc8e9063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:58.517389 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4723a70-acd3-431d-9076-4e9dfc8e9063 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:58.517663 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4723a70-acd3-431d-9076-4e9dfc8e9063 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:58.517933 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4723a70-acd3-431d-9076-4e9dfc8e9063 admin admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:58.518585 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4723a70-acd3-431d-9076-4e9dfc8e9063 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:20:58.518883 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4723a70-acd3-431d-9076-4e9dfc8e9063 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:58.520550 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4723a70-acd3-431d-9076-4e9dfc8e9063 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:58.532069 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdfa6a1-8e60-4d6b-85e7-586243a320ed None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:58.532772 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 862/1712] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Oct 12 00:20:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:20:58.541677 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d513a0-f829-4a98-b5e0-27739140dc7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:58.541677 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d513a0-f829-4a98-b5e0-27739140dc7a service cinder] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:58.541677 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d513a0-f829-4a98-b5e0-27739140dc7a service cinder] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:58.541677 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d513a0-f829-4a98-b5e0-27739140dc7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:58.542081 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4723a70-acd3-431d-9076-4e9dfc8e9063 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:58.548686 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4723a70-acd3-431d-9076-4e9dfc8e9063 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:58.552720 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d513a0-f829-4a98-b5e0-27739140dc7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:58.555614 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe6ac99-a291-450a-b29d-0f7ea8942f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:58.555895 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d513a0-f829-4a98-b5e0-27739140dc7a service cinder] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:58.556009 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe6ac99-a291-450a-b29d-0f7ea8942f2e admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:58.556009 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe6ac99-a291-450a-b29d-0f7ea8942f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:58.556009 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe6ac99-a291-450a-b29d-0f7ea8942f2e admin admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:58.556009 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe6ac99-a291-450a-b29d-0f7ea8942f2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:58.558100 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe6ac99-a291-450a-b29d-0f7ea8942f2e admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:58.564227 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 851/1713] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:20:58 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 00:20:58.581450 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d513a0-f829-4a98-b5e0-27739140dc7a service cinder] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:58.582831 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-06d513a0-f829-4a98-b5e0-27739140dc7a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:58.597652 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d513a0-f829-4a98-b5e0-27739140dc7a service cinder] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:58.602386 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22069c26-7e0d-4f99-9736-86db65795a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:58.603025 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22069c26-7e0d-4f99-9736-86db65795a2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:58.603400 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22069c26-7e0d-4f99-9736-86db65795a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:58.603653 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22069c26-7e0d-4f99-9736-86db65795a2f admin admin] PATH_INFO: `/v3/users/b84f16d32146489fb6dcab1b29c5bda9` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:58.604772 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22069c26-7e0d-4f99-9736-86db65795a2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b84f16d32146489fb6dcab1b29c5bda9)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:58.606703 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22069c26-7e0d-4f99-9736-86db65795a2f admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:58.653018 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 863/1714] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:20:58 2022] POST /identity/v3/projects => generated 391 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:20:58.657147 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe6ac99-a291-450a-b29d-0f7ea8942f2e admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:58.719584 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe6ac99-a291-450a-b29d-0f7ea8942f2e admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:58.724421 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051ac98b-deca-43b2-8fe2-5fc656ab6c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:58.725055 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051ac98b-deca-43b2-8fe2-5fc656ab6c4e admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:58.725331 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051ac98b-deca-43b2-8fe2-5fc656ab6c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:58.725593 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051ac98b-deca-43b2-8fe2-5fc656ab6c4e admin admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:58.726219 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051ac98b-deca-43b2-8fe2-5fc656ab6c4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:58.728181 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051ac98b-deca-43b2-8fe2-5fc656ab6c4e admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:58.818785 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-22069c26-7e0d-4f99-9736-86db65795a2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b84f16d32146489fb6dcab1b29c5bda9'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:58.819525 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-22069c26-7e0d-4f99-9736-86db65795a2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b84f16d32146489fb6dcab1b29c5bda9'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:58.831678 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-22069c26-7e0d-4f99-9736-86db65795a2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b84f16d32146489fb6dcab1b29c5bda9'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:58.839388 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 864/1715] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:20:58 2022] POST /identity/v3/users => generated 296 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 00:20:58.840402 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051ac98b-deca-43b2-8fe2-5fc656ab6c4e admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:58.842726 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-051ac98b-deca-43b2-8fe2-5fc656ab6c4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:58.899026 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051ac98b-deca-43b2-8fe2-5fc656ab6c4e admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:58.902838 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bade93eb-c497-4f7c-a060-9c20f1b3c003 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:58.903470 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade93eb-c497-4f7c-a060-9c20f1b3c003 service cinder] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:58.903748 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade93eb-c497-4f7c-a060-9c20f1b3c003 service cinder] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:58.904016 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade93eb-c497-4f7c-a060-9c20f1b3c003 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:58.905787 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-bade93eb-c497-4f7c-a060-9c20f1b3c003 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:58.924304 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-22069c26-7e0d-4f99-9736-86db65795a2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b84f16d32146489fb6dcab1b29c5bda9'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:58.924935 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-22069c26-7e0d-4f99-9736-86db65795a2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b84f16d32146489fb6dcab1b29c5bda9'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:58.938598 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 852/1716] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:20:58 2022] DELETE /identity/v3/users/b84f16d32146489fb6dcab1b29c5bda9 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:58.940182 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22069c26-7e0d-4f99-9736-86db65795a2f admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:58.941056 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-22069c26-7e0d-4f99-9736-86db65795a2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:58.972475 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bade93eb-c497-4f7c-a060-9c20f1b3c003 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:58.973834 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bade93eb-c497-4f7c-a060-9c20f1b3c003 service cinder] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:58.984135 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 865/1717] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:20:58 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:20:58.985548 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15df32f-0469-4b4b-b06b-4552ef95f4c1 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:58.985823 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15df32f-0469-4b4b-b06b-4552ef95f4c1 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:58.986097 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15df32f-0469-4b4b-b06b-4552ef95f4c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:59.005775 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-d15df32f-0469-4b4b-b06b-4552ef95f4c1 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:59.006988 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-d15df32f-0469-4b4b-b06b-4552ef95f4c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:59.009049 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22069c26-7e0d-4f99-9736-86db65795a2f admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:59.014124 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d52c5c-632f-4e90-8946-e766631fe26d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:59.014738 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d52c5c-632f-4e90-8946-e766631fe26d admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:59.015037 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d52c5c-632f-4e90-8946-e766631fe26d admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:59.015258 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 866/1718] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:59.018482 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d52c5c-632f-4e90-8946-e766631fe26d admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:59.018482 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d52c5c-632f-4e90-8946-e766631fe26d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:59.018482 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d52c5c-632f-4e90-8946-e766631fe26d admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:59.025149 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15df32f-0469-4b4b-b06b-4552ef95f4c1 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:59.026525 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-d15df32f-0469-4b4b-b06b-4552ef95f4c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:59.029650 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 853/1719] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:20:58 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:20:59.033685 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d52c5c-632f-4e90-8946-e766631fe26d admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:59.044086 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d52c5c-632f-4e90-8946-e766631fe26d admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:59.048774 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992f906b-0bab-4493-9740-442f01b1905b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:59.049365 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992f906b-0bab-4493-9740-442f01b1905b admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:59.049643 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992f906b-0bab-4493-9740-442f01b1905b admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:59.049914 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992f906b-0bab-4493-9740-442f01b1905b admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:59.050548 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992f906b-0bab-4493-9740-442f01b1905b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:59.052185 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992f906b-0bab-4493-9740-442f01b1905b admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:59.063095 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 854/1720] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:20:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 00:20:59.067756 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992f906b-0bab-4493-9740-442f01b1905b admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:59.077899 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992f906b-0bab-4493-9740-442f01b1905b admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:59.083512 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6ac59d-8d3e-4471-a967-7cd62c3fdfe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:59.084175 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ac59d-8d3e-4471-a967-7cd62c3fdfe0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:59.084452 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ac59d-8d3e-4471-a967-7cd62c3fdfe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:59.084725 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ac59d-8d3e-4471-a967-7cd62c3fdfe0 admin admin] PATH_INFO: `/v3/projects/28b2469901dd4280aa12effd7dfb3490/users/04104188d2214586b1cd6b72a355d80f/roles/ef689f96442945e8a7bc8cbfee13d909` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:59.089390 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15df32f-0469-4b4b-b06b-4552ef95f4c1 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:59.093178 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2d291b-e7d6-4767-be57-2dadc58ef06c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:59.098576 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2d291b-e7d6-4767-be57-2dadc58ef06c service neutron] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:59.098576 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2d291b-e7d6-4767-be57-2dadc58ef06c service neutron] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:59.098576 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2d291b-e7d6-4767-be57-2dadc58ef06c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:59.099688 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6ac59d-8d3e-4471-a967-7cd62c3fdfe0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28b2469901dd4280aa12effd7dfb3490, user_id=04104188d2214586b1cd6b72a355d80f, role_id=ef689f96442945e8a7bc8cbfee13d909)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:59.102335 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6ac59d-8d3e-4471-a967-7cd62c3fdfe0 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:59.103470 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2d291b-e7d6-4767-be57-2dadc58ef06c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:59.104949 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2d291b-e7d6-4767-be57-2dadc58ef06c service neutron] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:59.115502 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 867/1721] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:20:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 00:20:59.196489 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 855/1722] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:20:59 2022] PUT /identity/v3/projects/28b2469901dd4280aa12effd7dfb3490/users/04104188d2214586b1cd6b72a355d80f/roles/ef689f96442945e8a7bc8cbfee13d909 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:59.208845 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96410050-1ea3-43b2-9148-92788b14d6fa None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:59.209120 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96410050-1ea3-43b2-9148-92788b14d6fa None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:59.209386 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96410050-1ea3-43b2-9148-92788b14d6fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:59.240045 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-96410050-1ea3-43b2-9148-92788b14d6fa None None] MFA Rules not processed for user `04104188d2214586b1cd6b72a355d80f`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:20:59.241227 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-96410050-1ea3-43b2-9148-92788b14d6fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:59.309805 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 868/1723] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:59 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:20:59.314315 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6ac59d-8d3e-4471-a967-7cd62c3fdfe0 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:59.383870 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6ac59d-8d3e-4471-a967-7cd62c3fdfe0 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:59.389051 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761f583b-89aa-46bf-bccd-cea0e321b47a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:59.389664 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761f583b-89aa-46bf-bccd-cea0e321b47a admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:59.389947 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761f583b-89aa-46bf-bccd-cea0e321b47a admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:59.390210 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761f583b-89aa-46bf-bccd-cea0e321b47a admin admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:59.390891 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761f583b-89aa-46bf-bccd-cea0e321b47a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:59.393108 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761f583b-89aa-46bf-bccd-cea0e321b47a admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:59.394226 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:59.445098 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:20:59.489752 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 856/1724] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:20:59 2022] POST /identity/v3/projects => generated 393 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:20:59.490963 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96410050-1ea3-43b2-9148-92788b14d6fa None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:59.502056 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96410050-1ea3-43b2-9148-92788b14d6fa None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:59.567751 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619f17d2-d2c2-4af2-af86-09e99778840b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:59.568413 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f17d2-d2c2-4af2-af86-09e99778840b admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:59.568684 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f17d2-d2c2-4af2-af86-09e99778840b admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:59.568956 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f17d2-d2c2-4af2-af86-09e99778840b admin admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:59.569584 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619f17d2-d2c2-4af2-af86-09e99778840b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:59.571582 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619f17d2-d2c2-4af2-af86-09e99778840b admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:59.624302 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761f583b-89aa-46bf-bccd-cea0e321b47a admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:59.626054 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-761f583b-89aa-46bf-bccd-cea0e321b47a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:20:59.650249 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761f583b-89aa-46bf-bccd-cea0e321b47a admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:59.652674 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd621517-7db6-46b8-9901-daf3bc1db78c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:59.653415 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd621517-7db6-46b8-9901-daf3bc1db78c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:59.653754 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd621517-7db6-46b8-9901-daf3bc1db78c admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:59.654094 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd621517-7db6-46b8-9901-daf3bc1db78c admin admin] PATH_INFO: `/v3/projects/b5ce5a2e842041f3ab5a22f8fa519960` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:59.655636 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd621517-7db6-46b8-9901-daf3bc1db78c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5ce5a2e842041f3ab5a22f8fa519960)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:59.657851 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd621517-7db6-46b8-9901-daf3bc1db78c admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:59.680337 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 869/1725] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:20:59 2022] POST /identity/v3/users => generated 303 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:20:59.683933 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619f17d2-d2c2-4af2-af86-09e99778840b admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:59.705212 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619f17d2-d2c2-4af2-af86-09e99778840b admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:59.710279 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392bd77f-bdb8-4844-964b-60197cd2571b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:59.710891 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392bd77f-bdb8-4844-964b-60197cd2571b admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:59.711165 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392bd77f-bdb8-4844-964b-60197cd2571b admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:59.711461 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392bd77f-bdb8-4844-964b-60197cd2571b admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:59.712102 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392bd77f-bdb8-4844-964b-60197cd2571b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:59.713825 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392bd77f-bdb8-4844-964b-60197cd2571b admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:59.725205 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 870/1726] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:20:59 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 00:20:59.729375 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392bd77f-bdb8-4844-964b-60197cd2571b admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:59.739307 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392bd77f-bdb8-4844-964b-60197cd2571b admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:59.744467 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07876817-fdc9-4c44-8eed-d666f70624c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:59.745057 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07876817-fdc9-4c44-8eed-d666f70624c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:59.745335 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07876817-fdc9-4c44-8eed-d666f70624c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:59.745606 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07876817-fdc9-4c44-8eed-d666f70624c2 admin admin] PATH_INFO: `/v3/projects/ac3ce78eaddd422bb743324bc38babe8/users/0a13cb34deaf43ffbd1a2c3378feef70/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:59.760434 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07876817-fdc9-4c44-8eed-d666f70624c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac3ce78eaddd422bb743324bc38babe8, user_id=0a13cb34deaf43ffbd1a2c3378feef70, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:59.763988 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07876817-fdc9-4c44-8eed-d666f70624c2 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:59.859734 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 871/1727] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:20:59 2022] PUT /identity/v3/projects/ac3ce78eaddd422bb743324bc38babe8/users/0a13cb34deaf43ffbd1a2c3378feef70/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 00:20:59.863352 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07876817-fdc9-4c44-8eed-d666f70624c2 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:20:59.869835 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-dd621517-7db6-46b8-9901-daf3bc1db78c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5ce5a2e842041f3ab5a22f8fa519960', 'request_id': 'req-dd621517-7db6-46b8-9901-daf3bc1db78c'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:59.877509 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-dd621517-7db6-46b8-9901-daf3bc1db78c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5ce5a2e842041f3ab5a22f8fa519960', 'request_id': 'req-dd621517-7db6-46b8-9901-daf3bc1db78c'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:59.927789 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07876817-fdc9-4c44-8eed-d666f70624c2 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:20:59.958595 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-dd621517-7db6-46b8-9901-daf3bc1db78c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5ce5a2e842041f3ab5a22f8fa519960', 'request_id': 'req-dd621517-7db6-46b8-9901-daf3bc1db78c'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:59.970689 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-dd621517-7db6-46b8-9901-daf3bc1db78c admin admin] The token cache is being invalidate because project b5ce5a2e842041f3ab5a22f8fa519960 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81746) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:20:59.970983 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-dd621517-7db6-46b8-9901-daf3bc1db78c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:59.971654 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-dd621517-7db6-46b8-9901-daf3bc1db78c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:20:59.973484 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 857/1728] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:20:59 2022] DELETE /identity/v3/projects/b5ce5a2e842041f3ab5a22f8fa519960 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:20:59.978123 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54183ebf-f13e-4f95-b2cb-10a596f21275 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:59.978432 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54183ebf-f13e-4f95-b2cb-10a596f21275 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:59.978882 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54183ebf-f13e-4f95-b2cb-10a596f21275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:59.983631 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a20894-013a-4d4e-9569-7bc28f6546b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:20:59.984244 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a20894-013a-4d4e-9569-7bc28f6546b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:20:59.984517 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a20894-013a-4d4e-9569-7bc28f6546b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:20:59.984812 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a20894-013a-4d4e-9569-7bc28f6546b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:20:59.985442 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a20894-013a-4d4e-9569-7bc28f6546b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:20:59.987075 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a20894-013a-4d4e-9569-7bc28f6546b9 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:20:59.996951 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 872/1729] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:20:59 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 00:20:59.999307 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-54183ebf-f13e-4f95-b2cb-10a596f21275 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:21:00.000555 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-54183ebf-f13e-4f95-b2cb-10a596f21275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:00.001044 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a20894-013a-4d4e-9569-7bc28f6546b9 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:00.002689 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-f8a20894-013a-4d4e-9569-7bc28f6546b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:00.008857 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 858/1730] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:20:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 00:21:00.015565 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a20894-013a-4d4e-9569-7bc28f6546b9 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:00.020617 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3e6dc6-681f-4a30-8227-06dc35447c6c None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:00.020617 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3e6dc6-681f-4a30-8227-06dc35447c6c None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:00.020617 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3e6dc6-681f-4a30-8227-06dc35447c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:00.020874 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16191eea-cebd-4e7d-b729-2eed79e123c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:00.020874 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16191eea-cebd-4e7d-b729-2eed79e123c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:00.021052 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16191eea-cebd-4e7d-b729-2eed79e123c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:00.021202 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16191eea-cebd-4e7d-b729-2eed79e123c6 admin admin] PATH_INFO: `/v3/projects/ac3ce78eaddd422bb743324bc38babe8/users/0a13cb34deaf43ffbd1a2c3378feef70/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:00.023041 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16191eea-cebd-4e7d-b729-2eed79e123c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac3ce78eaddd422bb743324bc38babe8, user_id=0a13cb34deaf43ffbd1a2c3378feef70, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:00.025691 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16191eea-cebd-4e7d-b729-2eed79e123c6 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:00.036602 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 873/1731] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:21:00 2022] PUT /identity/v3/projects/ac3ce78eaddd422bb743324bc38babe8/users/0a13cb34deaf43ffbd1a2c3378feef70/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:21:00.037667 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-ac3e6dc6-681f-4a30-8227-06dc35447c6c None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:21:00.039336 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-ac3e6dc6-681f-4a30-8227-06dc35447c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:00.047127 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3967a1f3-3dfb-4d38-95f4-3aae88ba341b None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:00.047436 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3967a1f3-3dfb-4d38-95f4-3aae88ba341b None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:00.047707 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3967a1f3-3dfb-4d38-95f4-3aae88ba341b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:00.067941 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:00.075615 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-3967a1f3-3dfb-4d38-95f4-3aae88ba341b None None] MFA Rules not processed for user `0a13cb34deaf43ffbd1a2c3378feef70`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:21:00.076850 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-3967a1f3-3dfb-4d38-95f4-3aae88ba341b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:00.103130 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 859/1732] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:21:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 00:21:00.107441 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3e6dc6-681f-4a30-8227-06dc35447c6c None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:00.116250 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3e6dc6-681f-4a30-8227-06dc35447c6c None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:00.121826 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d61bbb3-ae56-426f-9f22-e4d065b3bbbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:00.122422 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d61bbb3-ae56-426f-9f22-e4d065b3bbbd admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:00.122699 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d61bbb3-ae56-426f-9f22-e4d065b3bbbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:00.122971 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d61bbb3-ae56-426f-9f22-e4d065b3bbbd admin admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:00.123754 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d61bbb3-ae56-426f-9f22-e4d065b3bbbd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:21:00.124104 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d61bbb3-ae56-426f-9f22-e4d065b3bbbd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:00.125716 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d61bbb3-ae56-426f-9f22-e4d065b3bbbd admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:00.137684 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 860/1733] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Oct 12 00:21:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 00:21:00.142081 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d61bbb3-ae56-426f-9f22-e4d065b3bbbd admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:00.151582 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 874/1734] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:21:00 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 00:21:00.158327 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3967a1f3-3dfb-4d38-95f4-3aae88ba341b None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:00.158968 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d61bbb3-ae56-426f-9f22-e4d065b3bbbd admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:00.159757 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-3967a1f3-3dfb-4d38-95f4-3aae88ba341b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:00.163970 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a30eaf-7e40-4b34-826f-5359c97922a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:00.164573 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a30eaf-7e40-4b34-826f-5359c97922a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:00.165082 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a30eaf-7e40-4b34-826f-5359c97922a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:00.165361 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a30eaf-7e40-4b34-826f-5359c97922a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:00.165997 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a30eaf-7e40-4b34-826f-5359c97922a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:00.168177 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a30eaf-7e40-4b34-826f-5359c97922a1 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:00.219995 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3967a1f3-3dfb-4d38-95f4-3aae88ba341b None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:00.223988 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9689243c-4427-430a-87dd-fc68ff79a93a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:00.224631 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9689243c-4427-430a-87dd-fc68ff79a93a service cinder] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:00.224909 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9689243c-4427-430a-87dd-fc68ff79a93a service cinder] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:00.225197 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9689243c-4427-430a-87dd-fc68ff79a93a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:00.234541 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9689243c-4427-430a-87dd-fc68ff79a93a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:00.236020 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9689243c-4427-430a-87dd-fc68ff79a93a service cinder] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:00.246258 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 875/1735] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:21:00 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 00:21:00.283360 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 861/1736] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:21:00 2022] POST /identity/v3/projects => generated 383 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 00:21:00.284517 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9689243c-4427-430a-87dd-fc68ff79a93a service cinder] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:00.294412 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a30eaf-7e40-4b34-826f-5359c97922a1 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:00.295579 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9689243c-4427-430a-87dd-fc68ff79a93a service cinder] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:00.296208 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-a9a30eaf-7e40-4b34-826f-5359c97922a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:00.353536 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f28951-4b84-40e6-80e0-0af3a47a1b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:00.354275 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f28951-4b84-40e6-80e0-0af3a47a1b76 admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:00.354554 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f28951-4b84-40e6-80e0-0af3a47a1b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:00.354824 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f28951-4b84-40e6-80e0-0af3a47a1b76 admin admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:00.355499 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f28951-4b84-40e6-80e0-0af3a47a1b76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:00.357561 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f28951-4b84-40e6-80e0-0af3a47a1b76 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:00.357803 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a30eaf-7e40-4b34-826f-5359c97922a1 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:00.361501 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6022af18-23bd-417f-8460-d378798a8e12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:00.362106 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6022af18-23bd-417f-8460-d378798a8e12 service cinder] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:00.362385 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6022af18-23bd-417f-8460-d378798a8e12 service cinder] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:00.363521 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6022af18-23bd-417f-8460-d378798a8e12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:00.364554 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-6022af18-23bd-417f-8460-d378798a8e12 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:00.421792 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6022af18-23bd-417f-8460-d378798a8e12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:00.423224 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6022af18-23bd-417f-8460-d378798a8e12 service cinder] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:00.431882 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 862/1737] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:21:00 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 00:21:00.450763 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 876/1738] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:21:00 2022] POST /identity/v3/users => generated 292 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 00:21:00.454942 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6022af18-23bd-417f-8460-d378798a8e12 service cinder] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:00.465609 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6022af18-23bd-417f-8460-d378798a8e12 service cinder] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:00.470284 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937d9080-31c8-4987-96d6-7bf8ce4dd8a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:00.470883 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937d9080-31c8-4987-96d6-7bf8ce4dd8a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:00.471182 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937d9080-31c8-4987-96d6-7bf8ce4dd8a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:00.471493 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937d9080-31c8-4987-96d6-7bf8ce4dd8a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:00.472135 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937d9080-31c8-4987-96d6-7bf8ce4dd8a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:00.473750 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937d9080-31c8-4987-96d6-7bf8ce4dd8a4 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:00.484715 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 863/1739] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:21:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:21:00.488801 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f28951-4b84-40e6-80e0-0af3a47a1b76 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:00.501569 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f28951-4b84-40e6-80e0-0af3a47a1b76 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:00.506479 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6540da-2351-4191-acfa-d885e2d5fe3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:00.507161 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6540da-2351-4191-acfa-d885e2d5fe3a admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:00.507475 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6540da-2351-4191-acfa-d885e2d5fe3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:00.512710 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6540da-2351-4191-acfa-d885e2d5fe3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:00.512710 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6540da-2351-4191-acfa-d885e2d5fe3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:00.512710 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6540da-2351-4191-acfa-d885e2d5fe3a admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:00.529582 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 877/1740] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:21:00 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:21:00.530718 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937d9080-31c8-4987-96d6-7bf8ce4dd8a4 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:00.532833 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:00.553350 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937d9080-31c8-4987-96d6-7bf8ce4dd8a4 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:00.559071 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7036d25-993c-41ed-9c3b-8cce8f16d919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:00.559976 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7036d25-993c-41ed-9c3b-8cce8f16d919 admin admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:00.560523 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7036d25-993c-41ed-9c3b-8cce8f16d919 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:00.560929 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7036d25-993c-41ed-9c3b-8cce8f16d919 admin admin] PATH_INFO: `/v3/projects/43a2f7ca74f54c289c1862ab5fb2163f/users/af2e7c0875e942eaa03113a2d5f9ea19/roles/ef689f96442945e8a7bc8cbfee13d909` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:00.591737 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7036d25-993c-41ed-9c3b-8cce8f16d919 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43a2f7ca74f54c289c1862ab5fb2163f, user_id=af2e7c0875e942eaa03113a2d5f9ea19, role_id=ef689f96442945e8a7bc8cbfee13d909)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:00.594540 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7036d25-993c-41ed-9c3b-8cce8f16d919 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:00.699035 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 864/1741] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:21:00 2022] PUT /identity/v3/projects/43a2f7ca74f54c289c1862ab5fb2163f/users/af2e7c0875e942eaa03113a2d5f9ea19/roles/ef689f96442945e8a7bc8cbfee13d909 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:21:00.709327 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5649a0-cc99-48bf-950f-e72529d2fa93 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:00.709608 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5649a0-cc99-48bf-950f-e72529d2fa93 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:00.709895 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5649a0-cc99-48bf-950f-e72529d2fa93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:00.738888 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-2a5649a0-cc99-48bf-950f-e72529d2fa93 None None] MFA Rules not processed for user `af2e7c0875e942eaa03113a2d5f9ea19`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:21:00.740172 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-2a5649a0-cc99-48bf-950f-e72529d2fa93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:00.812324 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 878/1742] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:21:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:21:00.816755 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7036d25-993c-41ed-9c3b-8cce8f16d919 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:00.886986 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7036d25-993c-41ed-9c3b-8cce8f16d919 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:00.891985 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d7873c-9abb-4f7e-93f0-d6e8283e05e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:00.892600 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d7873c-9abb-4f7e-93f0-d6e8283e05e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:00.892877 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d7873c-9abb-4f7e-93f0-d6e8283e05e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:00.893144 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d7873c-9abb-4f7e-93f0-d6e8283e05e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:00.893770 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d7873c-9abb-4f7e-93f0-d6e8283e05e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:00.895692 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d7873c-9abb-4f7e-93f0-d6e8283e05e8 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:01.038560 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 865/1743] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:21:00 2022] POST /identity/v3/projects => generated 381 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:21:01.043422 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5649a0-cc99-48bf-950f-e72529d2fa93 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:01.054122 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5649a0-cc99-48bf-950f-e72529d2fa93 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:01.109239 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01809b1e-7777-4cca-801e-bb8db45b5571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:01.109854 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01809b1e-7777-4cca-801e-bb8db45b5571 admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:01.110131 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01809b1e-7777-4cca-801e-bb8db45b5571 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:01.110401 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01809b1e-7777-4cca-801e-bb8db45b5571 admin admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:01.111033 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01809b1e-7777-4cca-801e-bb8db45b5571 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:01.112984 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01809b1e-7777-4cca-801e-bb8db45b5571 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:01.250955 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 879/1744] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:21:01 2022] POST /identity/v3/users => generated 291 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:21:01.252328 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d7873c-9abb-4f7e-93f0-d6e8283e05e8 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:01.268452 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d7873c-9abb-4f7e-93f0-d6e8283e05e8 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:01.280757 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844c8c09-ce22-447d-a582-d872782e05e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:01.281337 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844c8c09-ce22-447d-a582-d872782e05e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:01.281617 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844c8c09-ce22-447d-a582-d872782e05e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:01.281914 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844c8c09-ce22-447d-a582-d872782e05e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:01.282567 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844c8c09-ce22-447d-a582-d872782e05e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:01.284331 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844c8c09-ce22-447d-a582-d872782e05e8 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:01.300429 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 866/1745] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:21:01 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:21:01.304705 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844c8c09-ce22-447d-a582-d872782e05e8 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:01.305283 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01809b1e-7777-4cca-801e-bb8db45b5571 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:01.306476 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-844c8c09-ce22-447d-a582-d872782e05e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:01.317717 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01809b1e-7777-4cca-801e-bb8db45b5571 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:01.322828 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844c8c09-ce22-447d-a582-d872782e05e8 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:01.323380 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5747ce01-2d7c-4797-a760-cb881fd34fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:01.324546 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5747ce01-2d7c-4797-a760-cb881fd34fde admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:01.324819 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5747ce01-2d7c-4797-a760-cb881fd34fde admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:01.325179 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5747ce01-2d7c-4797-a760-cb881fd34fde admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:01.325886 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5747ce01-2d7c-4797-a760-cb881fd34fde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:01.327700 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5747ce01-2d7c-4797-a760-cb881fd34fde admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:01.329626 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca82b0c-ffe3-41cf-b375-52d91dbb614d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:01.330835 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca82b0c-ffe3-41cf-b375-52d91dbb614d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:01.331214 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca82b0c-ffe3-41cf-b375-52d91dbb614d admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:01.331714 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca82b0c-ffe3-41cf-b375-52d91dbb614d admin admin] PATH_INFO: `/v3/users/77a415a0c886456695ecb014b0ffbdd7` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:01.333203 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca82b0c-ffe3-41cf-b375-52d91dbb614d admin admin] RBAC: Authorizing `identity:delete_user(user_id=77a415a0c886456695ecb014b0ffbdd7)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:01.335797 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca82b0c-ffe3-41cf-b375-52d91dbb614d admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:01.341458 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 880/1746] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:21:01 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:21:01.345886 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5747ce01-2d7c-4797-a760-cb881fd34fde admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:01.363301 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5747ce01-2d7c-4797-a760-cb881fd34fde admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:01.368484 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb512163-a76a-43cb-91c2-456ae6ee819d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:01.369152 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb512163-a76a-43cb-91c2-456ae6ee819d admin admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:01.371934 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb512163-a76a-43cb-91c2-456ae6ee819d admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:01.371934 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb512163-a76a-43cb-91c2-456ae6ee819d admin admin] PATH_INFO: `/v3/projects/a5d529eda9854769b44da851caed0ace/users/90457bdd8d8f4c39a57e190675e9ad65/roles/ef689f96442945e8a7bc8cbfee13d909` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:01.384844 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb512163-a76a-43cb-91c2-456ae6ee819d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5d529eda9854769b44da851caed0ace, user_id=90457bdd8d8f4c39a57e190675e9ad65, role_id=ef689f96442945e8a7bc8cbfee13d909)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:01.388500 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb512163-a76a-43cb-91c2-456ae6ee819d admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:01.477270 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 881/1747] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:21:01 2022] PUT /identity/v3/projects/a5d529eda9854769b44da851caed0ace/users/90457bdd8d8f4c39a57e190675e9ad65/roles/ef689f96442945e8a7bc8cbfee13d909 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:21:01.488543 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c922cebe-c1ce-4acc-bba3-db119170018d None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:01.488823 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c922cebe-c1ce-4acc-bba3-db119170018d None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:01.489118 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c922cebe-c1ce-4acc-bba3-db119170018d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:01.518526 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-c922cebe-c1ce-4acc-bba3-db119170018d None None] MFA Rules not processed for user `90457bdd8d8f4c39a57e190675e9ad65`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:21:01.524914 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-c922cebe-c1ce-4acc-bba3-db119170018d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:01.586241 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 882/1748] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:21:01 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 00:21:01.591445 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c922cebe-c1ce-4acc-bba3-db119170018d None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:01.591681 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-cca82b0c-ffe3-41cf-b375-52d91dbb614d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77a415a0c886456695ecb014b0ffbdd7'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:01.592365 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-cca82b0c-ffe3-41cf-b375-52d91dbb614d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77a415a0c886456695ecb014b0ffbdd7'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:01.604580 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-cca82b0c-ffe3-41cf-b375-52d91dbb614d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77a415a0c886456695ecb014b0ffbdd7'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:01.653482 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c922cebe-c1ce-4acc-bba3-db119170018d None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:01.658305 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5e5961-bd48-446c-950a-f6dcf90f9cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:01.658917 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5e5961-bd48-446c-950a-f6dcf90f9cf4 admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:01.659197 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5e5961-bd48-446c-950a-f6dcf90f9cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:01.659550 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5e5961-bd48-446c-950a-f6dcf90f9cf4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:01.660180 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5e5961-bd48-446c-950a-f6dcf90f9cf4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:01.662121 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5e5961-bd48-446c-950a-f6dcf90f9cf4 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:01.688817 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-cca82b0c-ffe3-41cf-b375-52d91dbb614d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77a415a0c886456695ecb014b0ffbdd7'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:01.689761 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-cca82b0c-ffe3-41cf-b375-52d91dbb614d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77a415a0c886456695ecb014b0ffbdd7'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:01.704097 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 867/1749] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:21:01 2022] DELETE /identity/v3/users/77a415a0c886456695ecb014b0ffbdd7 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:21:01.708501 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6067b3-d3cc-4975-9d64-998b31cbb750 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:01.708787 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6067b3-d3cc-4975-9d64-998b31cbb750 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:01.709127 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6067b3-d3cc-4975-9d64-998b31cbb750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:01.728767 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-1f6067b3-d3cc-4975-9d64-998b31cbb750 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:21:01.730030 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-1f6067b3-d3cc-4975-9d64-998b31cbb750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:01.763891 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 883/1750] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:21:01 2022] POST /identity/v3/projects => generated 383 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 00:21:01.767832 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5e5961-bd48-446c-950a-f6dcf90f9cf4 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:01.769646 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-8c5e5961-bd48-446c-950a-f6dcf90f9cf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:01.793252 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 868/1751] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:21:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:21:01.809075 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6067b3-d3cc-4975-9d64-998b31cbb750 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:01.810470 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-1f6067b3-d3cc-4975-9d64-998b31cbb750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:01.835252 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5e5961-bd48-446c-950a-f6dcf90f9cf4 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:01.840846 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ece2ad9-ccd2-4830-b281-660542afb944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:01.841455 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ece2ad9-ccd2-4830-b281-660542afb944 admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:01.842103 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ece2ad9-ccd2-4830-b281-660542afb944 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:01.842196 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ece2ad9-ccd2-4830-b281-660542afb944 admin admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:01.842908 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ece2ad9-ccd2-4830-b281-660542afb944 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:01.844762 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ece2ad9-ccd2-4830-b281-660542afb944 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:01.871213 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6067b3-d3cc-4975-9d64-998b31cbb750 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:01.881702 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:01.919334 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcbcbd5-7b13-4753-862a-2d8d195c89a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:01.919972 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcbcbd5-7b13-4753-862a-2d8d195c89a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:01.920248 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcbcbd5-7b13-4753-862a-2d8d195c89a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:01.920515 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcbcbd5-7b13-4753-862a-2d8d195c89a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:01.929590 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcbcbd5-7b13-4753-862a-2d8d195c89a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:01.931026 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcbcbd5-7b13-4753-862a-2d8d195c89a3 service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:01.941308 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 869/1752] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:21:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 00:21:01.991448 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 884/1753] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:21:01 2022] POST /identity/v3/users => generated 298 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 00:21:01.993116 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcbcbd5-7b13-4753-862a-2d8d195c89a3 service neutron] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:02.004164 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcbcbd5-7b13-4753-862a-2d8d195c89a3 service neutron] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:02.010144 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b17e09c-e476-40b3-ae07-aa731018a0ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:02.010922 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b17e09c-e476-40b3-ae07-aa731018a0ac admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:02.011154 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b17e09c-e476-40b3-ae07-aa731018a0ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:02.011467 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b17e09c-e476-40b3-ae07-aa731018a0ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:02.012214 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b17e09c-e476-40b3-ae07-aa731018a0ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:02.014222 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b17e09c-e476-40b3-ae07-aa731018a0ac admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:02.027133 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 870/1754] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:21:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:21:02.031758 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ece2ad9-ccd2-4830-b281-660542afb944 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:02.055146 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ece2ad9-ccd2-4830-b281-660542afb944 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:02.060654 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b447efd9-02d3-4cf4-b958-932603075f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:02.061324 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b447efd9-02d3-4cf4-b958-932603075f24 admin admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:02.061601 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b447efd9-02d3-4cf4-b958-932603075f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:02.061875 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b447efd9-02d3-4cf4-b958-932603075f24 admin admin] PATH_INFO: `/v3/projects/246401c287454d22b0dc653ed0a65525/users/924bf25903a848b8a888f1f2ec9453dc/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:02.076917 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b447efd9-02d3-4cf4-b958-932603075f24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=246401c287454d22b0dc653ed0a65525, user_id=924bf25903a848b8a888f1f2ec9453dc, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:02.093575 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b447efd9-02d3-4cf4-b958-932603075f24 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:02.141556 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:02.181886 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 885/1755] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:21:02 2022] PUT /identity/v3/projects/246401c287454d22b0dc653ed0a65525/users/924bf25903a848b8a888f1f2ec9453dc/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:21:02.186441 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b17e09c-e476-40b3-ae07-aa731018a0ac admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:02.204754 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b17e09c-e476-40b3-ae07-aa731018a0ac admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:02.275954 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b538ba86-4e65-4b1a-b78c-a077811218fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:02.276795 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b538ba86-4e65-4b1a-b78c-a077811218fe admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:02.277722 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b538ba86-4e65-4b1a-b78c-a077811218fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:02.278146 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b538ba86-4e65-4b1a-b78c-a077811218fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:02.278942 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b538ba86-4e65-4b1a-b78c-a077811218fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:02.281530 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b538ba86-4e65-4b1a-b78c-a077811218fe admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:02.294230 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 871/1756] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:21:02 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:21:02.309251 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b447efd9-02d3-4cf4-b958-932603075f24 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:02.329551 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b447efd9-02d3-4cf4-b958-932603075f24 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:02.337114 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac844065-7563-4dfe-9f7a-b60dc58c79c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:02.340270 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac844065-7563-4dfe-9f7a-b60dc58c79c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:02.340270 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac844065-7563-4dfe-9f7a-b60dc58c79c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:02.340270 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac844065-7563-4dfe-9f7a-b60dc58c79c9 admin admin] PATH_INFO: `/v3/projects/246401c287454d22b0dc653ed0a65525/users/924bf25903a848b8a888f1f2ec9453dc/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:02.342396 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac844065-7563-4dfe-9f7a-b60dc58c79c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=246401c287454d22b0dc653ed0a65525, user_id=924bf25903a848b8a888f1f2ec9453dc, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:02.346698 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac844065-7563-4dfe-9f7a-b60dc58c79c9 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:02.362808 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 886/1757] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:21:02 2022] PUT /identity/v3/projects/246401c287454d22b0dc653ed0a65525/users/924bf25903a848b8a888f1f2ec9453dc/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:21:02.373744 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbce2f9e-700a-48dd-9f08-89a59adfab69 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:02.374764 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbce2f9e-700a-48dd-9f08-89a59adfab69 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:02.375187 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbce2f9e-700a-48dd-9f08-89a59adfab69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:02.422749 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-bbce2f9e-700a-48dd-9f08-89a59adfab69 None None] MFA Rules not processed for user `924bf25903a848b8a888f1f2ec9453dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:21:02.424349 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-bbce2f9e-700a-48dd-9f08-89a59adfab69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:02.505349 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 872/1758] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:21:02 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:21:02.510822 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac844065-7563-4dfe-9f7a-b60dc58c79c9 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:02.512624 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-ac844065-7563-4dfe-9f7a-b60dc58c79c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:02.518618 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbce2f9e-700a-48dd-9f08-89a59adfab69 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:02.520299 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-bbce2f9e-700a-48dd-9f08-89a59adfab69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:02.642143 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbce2f9e-700a-48dd-9f08-89a59adfab69 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:02.643259 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac844065-7563-4dfe-9f7a-b60dc58c79c9 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:02.647962 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24f7e75-113a-432f-99be-4a4896dd255e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:02.649054 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f7e75-113a-432f-99be-4a4896dd255e service cinder] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:02.649515 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f7e75-113a-432f-99be-4a4896dd255e service cinder] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:02.649860 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f7e75-113a-432f-99be-4a4896dd255e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:02.650786 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b26b62-ad99-4d7e-8336-2b86f0a5251e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:02.652230 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b26b62-ad99-4d7e-8336-2b86f0a5251e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:02.652562 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b26b62-ad99-4d7e-8336-2b86f0a5251e admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:02.652840 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b26b62-ad99-4d7e-8336-2b86f0a5251e admin admin] PATH_INFO: `/v3/projects/879b48942542406ba8df21971f4174cb` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:02.654212 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b26b62-ad99-4d7e-8336-2b86f0a5251e admin admin] RBAC: Authorizing `identity:delete_project(project_id=879b48942542406ba8df21971f4174cb)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:02.654692 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-d24f7e75-113a-432f-99be-4a4896dd255e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:02.656646 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b26b62-ad99-4d7e-8336-2b86f0a5251e admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:02.723831 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24f7e75-113a-432f-99be-4a4896dd255e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:02.725264 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24f7e75-113a-432f-99be-4a4896dd255e service cinder] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:02.734947 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 873/1759] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:21:02 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 00:21:02.819453 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-16b26b62-ad99-4d7e-8336-2b86f0a5251e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '879b48942542406ba8df21971f4174cb', 'request_id': 'req-16b26b62-ad99-4d7e-8336-2b86f0a5251e'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:02.827700 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-16b26b62-ad99-4d7e-8336-2b86f0a5251e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '879b48942542406ba8df21971f4174cb', 'request_id': 'req-16b26b62-ad99-4d7e-8336-2b86f0a5251e'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:02.924851 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-16b26b62-ad99-4d7e-8336-2b86f0a5251e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '879b48942542406ba8df21971f4174cb', 'request_id': 'req-16b26b62-ad99-4d7e-8336-2b86f0a5251e'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:02.940990 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-16b26b62-ad99-4d7e-8336-2b86f0a5251e admin admin] The token cache is being invalidate because project 879b48942542406ba8df21971f4174cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81747) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:21:02.941089 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-16b26b62-ad99-4d7e-8336-2b86f0a5251e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:02.941805 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-16b26b62-ad99-4d7e-8336-2b86f0a5251e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:02.943394 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 887/1760] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:21:02 2022] DELETE /identity/v3/projects/879b48942542406ba8df21971f4174cb => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:21:02.947613 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24f7e75-113a-432f-99be-4a4896dd255e service cinder] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:02.961001 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24f7e75-113a-432f-99be-4a4896dd255e service cinder] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:03.025749 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36520356-e702-4eec-9893-990845fe60e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:03.026522 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36520356-e702-4eec-9893-990845fe60e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:03.027530 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36520356-e702-4eec-9893-990845fe60e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:03.027811 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36520356-e702-4eec-9893-990845fe60e6 admin admin] PATH_INFO: `/v3/users/2fd2cf0fe8c54f5b9398bfcd8942bb0f` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:03.028958 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36520356-e702-4eec-9893-990845fe60e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fd2cf0fe8c54f5b9398bfcd8942bb0f)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:03.030935 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36520356-e702-4eec-9893-990845fe60e6 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:03.091288 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:03.239109 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-36520356-e702-4eec-9893-990845fe60e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fd2cf0fe8c54f5b9398bfcd8942bb0f'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:03.239918 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-36520356-e702-4eec-9893-990845fe60e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fd2cf0fe8c54f5b9398bfcd8942bb0f'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:03.253963 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-36520356-e702-4eec-9893-990845fe60e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fd2cf0fe8c54f5b9398bfcd8942bb0f'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:03.313020 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-36520356-e702-4eec-9893-990845fe60e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fd2cf0fe8c54f5b9398bfcd8942bb0f'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:03.313682 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-36520356-e702-4eec-9893-990845fe60e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fd2cf0fe8c54f5b9398bfcd8942bb0f'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:03.334583 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 874/1761] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:21:02 2022] DELETE /identity/v3/users/2fd2cf0fe8c54f5b9398bfcd8942bb0f => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 00:21:03.473728 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:03.738805 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:03.782827 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b26b62-ad99-4d7e-8336-2b86f0a5251e admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:03.784331 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-16b26b62-ad99-4d7e-8336-2b86f0a5251e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:03.800807 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b26b62-ad99-4d7e-8336-2b86f0a5251e admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:03.864921 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc84d5ff-0220-4277-b1d3-65825b6dc777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:03.865608 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc84d5ff-0220-4277-b1d3-65825b6dc777 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:03.865907 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc84d5ff-0220-4277-b1d3-65825b6dc777 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:03.869644 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc84d5ff-0220-4277-b1d3-65825b6dc777 admin admin] PATH_INFO: `/v3/projects/5028d039086546f3a267f246a6734656` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:03.869644 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc84d5ff-0220-4277-b1d3-65825b6dc777 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5028d039086546f3a267f246a6734656)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:03.870011 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc84d5ff-0220-4277-b1d3-65825b6dc777 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:03.973292 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-cc84d5ff-0220-4277-b1d3-65825b6dc777 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5028d039086546f3a267f246a6734656', 'request_id': 'req-cc84d5ff-0220-4277-b1d3-65825b6dc777'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:03.981956 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-cc84d5ff-0220-4277-b1d3-65825b6dc777 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5028d039086546f3a267f246a6734656', 'request_id': 'req-cc84d5ff-0220-4277-b1d3-65825b6dc777'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:04.055722 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-cc84d5ff-0220-4277-b1d3-65825b6dc777 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5028d039086546f3a267f246a6734656', 'request_id': 'req-cc84d5ff-0220-4277-b1d3-65825b6dc777'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:04.066775 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-cc84d5ff-0220-4277-b1d3-65825b6dc777 admin admin] The token cache is being invalidate because project 5028d039086546f3a267f246a6734656 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81747) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:21:04.067055 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-cc84d5ff-0220-4277-b1d3-65825b6dc777 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:04.067683 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-cc84d5ff-0220-4277-b1d3-65825b6dc777 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:04.069169 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 888/1762] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:21:03 2022] DELETE /identity/v3/projects/5028d039086546f3a267f246a6734656 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:21:04.073372 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36520356-e702-4eec-9893-990845fe60e6 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:04.074792 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-36520356-e702-4eec-9893-990845fe60e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:04.088173 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36520356-e702-4eec-9893-990845fe60e6 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:04.146451 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0e3cf4-9dbc-42e8-8fba-ee34c2b7094d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:04.147013 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0e3cf4-9dbc-42e8-8fba-ee34c2b7094d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:04.147292 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0e3cf4-9dbc-42e8-8fba-ee34c2b7094d admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:04.147662 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0e3cf4-9dbc-42e8-8fba-ee34c2b7094d admin admin] PATH_INFO: `/v3/users/bd3967ff06eb4f18bdebd396dbfa2d65` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:04.148786 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0e3cf4-9dbc-42e8-8fba-ee34c2b7094d admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd3967ff06eb4f18bdebd396dbfa2d65)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:04.150712 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0e3cf4-9dbc-42e8-8fba-ee34c2b7094d admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:04.428448 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-fb0e3cf4-9dbc-42e8-8fba-ee34c2b7094d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd3967ff06eb4f18bdebd396dbfa2d65'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:04.429246 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-fb0e3cf4-9dbc-42e8-8fba-ee34c2b7094d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd3967ff06eb4f18bdebd396dbfa2d65'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:04.441176 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-fb0e3cf4-9dbc-42e8-8fba-ee34c2b7094d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd3967ff06eb4f18bdebd396dbfa2d65'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:04.470069 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:04.538358 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-fb0e3cf4-9dbc-42e8-8fba-ee34c2b7094d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd3967ff06eb4f18bdebd396dbfa2d65'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:04.538988 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-fb0e3cf4-9dbc-42e8-8fba-ee34c2b7094d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd3967ff06eb4f18bdebd396dbfa2d65'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:04.552313 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 875/1763] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:21:04 2022] DELETE /identity/v3/users/bd3967ff06eb4f18bdebd396dbfa2d65 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 00:21:04.911234 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc84d5ff-0220-4277-b1d3-65825b6dc777 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:04.911234 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-cc84d5ff-0220-4277-b1d3-65825b6dc777 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:04.970488 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc84d5ff-0220-4277-b1d3-65825b6dc777 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:05.019187 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f389a2e1-6f32-4f7d-82da-e4307dfea6f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:05.019860 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f389a2e1-6f32-4f7d-82da-e4307dfea6f8 service cinder] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:05.020154 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f389a2e1-6f32-4f7d-82da-e4307dfea6f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:05.020430 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f389a2e1-6f32-4f7d-82da-e4307dfea6f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:05.022177 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-f389a2e1-6f32-4f7d-82da-e4307dfea6f8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:05.057958 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0e3cf4-9dbc-42e8-8fba-ee34c2b7094d admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:05.059987 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-fb0e3cf4-9dbc-42e8-8fba-ee34c2b7094d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:05.085681 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f389a2e1-6f32-4f7d-82da-e4307dfea6f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:05.087137 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f389a2e1-6f32-4f7d-82da-e4307dfea6f8 service cinder] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:05.099544 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 889/1764] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:21:04 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:21:05.141516 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0e3cf4-9dbc-42e8-8fba-ee34c2b7094d admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:05.146383 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44bcba7-f9c9-4545-8540-497c911aa4c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:05.147015 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44bcba7-f9c9-4545-8540-497c911aa4c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:05.147289 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44bcba7-f9c9-4545-8540-497c911aa4c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:05.147583 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44bcba7-f9c9-4545-8540-497c911aa4c4 admin admin] PATH_INFO: `/v3/projects/9aef5295bd4949a8b2cb6478e578e569` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:05.148773 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44bcba7-f9c9-4545-8540-497c911aa4c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9aef5295bd4949a8b2cb6478e578e569)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:05.150789 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44bcba7-f9c9-4545-8540-497c911aa4c4 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:05.257348 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-d44bcba7-f9c9-4545-8540-497c911aa4c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9aef5295bd4949a8b2cb6478e578e569', 'request_id': 'req-d44bcba7-f9c9-4545-8540-497c911aa4c4'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:05.264394 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-d44bcba7-f9c9-4545-8540-497c911aa4c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9aef5295bd4949a8b2cb6478e578e569', 'request_id': 'req-d44bcba7-f9c9-4545-8540-497c911aa4c4'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:05.335536 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-d44bcba7-f9c9-4545-8540-497c911aa4c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9aef5295bd4949a8b2cb6478e578e569', 'request_id': 'req-d44bcba7-f9c9-4545-8540-497c911aa4c4'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:05.348336 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-d44bcba7-f9c9-4545-8540-497c911aa4c4 admin admin] The token cache is being invalidate because project 9aef5295bd4949a8b2cb6478e578e569 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81746) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:21:05.348627 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-d44bcba7-f9c9-4545-8540-497c911aa4c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:05.349269 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-d44bcba7-f9c9-4545-8540-497c911aa4c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:05.351522 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 876/1765] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:21:05 2022] DELETE /identity/v3/projects/9aef5295bd4949a8b2cb6478e578e569 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 00:21:05.356954 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65c3f35-8f25-4ace-81a8-a49de19ff467 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:05.357237 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65c3f35-8f25-4ace-81a8-a49de19ff467 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:05.357514 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65c3f35-8f25-4ace-81a8-a49de19ff467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:05.377298 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-b65c3f35-8f25-4ace-81a8-a49de19ff467 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:21:05.378493 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-b65c3f35-8f25-4ace-81a8-a49de19ff467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:05.393335 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:05.441877 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 890/1766] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:21:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:21:05.450001 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a4aa90-f82e-453c-843d-740c8188f6c5 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:05.450439 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a4aa90-f82e-453c-843d-740c8188f6c5 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:05.450439 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a4aa90-f82e-453c-843d-740c8188f6c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:05.472090 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-a7a4aa90-f82e-453c-843d-740c8188f6c5 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:21:05.473757 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-a7a4aa90-f82e-453c-843d-740c8188f6c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:05.483740 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 877/1767] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:21:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:21:05.488300 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65c3f35-8f25-4ace-81a8-a49de19ff467 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:05.498814 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65c3f35-8f25-4ace-81a8-a49de19ff467 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:05.504122 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dc8769-508a-481c-84ea-ba987076e56f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:05.504747 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dc8769-508a-481c-84ea-ba987076e56f admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:05.505023 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dc8769-508a-481c-84ea-ba987076e56f admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:05.505295 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dc8769-508a-481c-84ea-ba987076e56f admin admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:05.506555 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dc8769-508a-481c-84ea-ba987076e56f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:21:05.506555 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dc8769-508a-481c-84ea-ba987076e56f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:05.508176 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dc8769-508a-481c-84ea-ba987076e56f admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:05.523412 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 891/1768] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Oct 12 00:21:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 00:21:05.532437 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a4aa90-f82e-453c-843d-740c8188f6c5 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:05.557432 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a4aa90-f82e-453c-843d-740c8188f6c5 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:05.562706 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a9cc0f-00c1-41f6-976d-1c0d0bb0adea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:05.563583 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a9cc0f-00c1-41f6-976d-1c0d0bb0adea admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:05.564023 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a9cc0f-00c1-41f6-976d-1c0d0bb0adea admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:05.564491 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a9cc0f-00c1-41f6-976d-1c0d0bb0adea admin admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:05.565312 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a9cc0f-00c1-41f6-976d-1c0d0bb0adea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:05.567486 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a9cc0f-00c1-41f6-976d-1c0d0bb0adea admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:05.653280 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:05.726700 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 878/1769] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:21:05 2022] POST /identity/v3/projects => generated 381 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 00:21:05.728200 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dc8769-508a-481c-84ea-ba987076e56f admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:05.738255 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dc8769-508a-481c-84ea-ba987076e56f admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:05.796660 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abca56b-e50e-436e-ae7c-2d734992eb2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:05.797274 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abca56b-e50e-436e-ae7c-2d734992eb2c admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:05.797548 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abca56b-e50e-436e-ae7c-2d734992eb2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:05.797816 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abca56b-e50e-436e-ae7c-2d734992eb2c admin admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:05.798441 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abca56b-e50e-436e-ae7c-2d734992eb2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:05.800400 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abca56b-e50e-436e-ae7c-2d734992eb2c admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:05.921840 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 892/1770] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:21:05 2022] POST /identity/v3/users => generated 291 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:21:05.923138 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a9cc0f-00c1-41f6-976d-1c0d0bb0adea admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:05.933590 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a9cc0f-00c1-41f6-976d-1c0d0bb0adea admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:05.938295 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08211b8c-646d-48cf-bcae-3b4aed504a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:05.938899 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08211b8c-646d-48cf-bcae-3b4aed504a94 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:05.939185 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08211b8c-646d-48cf-bcae-3b4aed504a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:05.939501 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08211b8c-646d-48cf-bcae-3b4aed504a94 admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:05.940171 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08211b8c-646d-48cf-bcae-3b4aed504a94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:05.941789 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08211b8c-646d-48cf-bcae-3b4aed504a94 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:05.955452 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 879/1771] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:21:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:21:05.956797 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abca56b-e50e-436e-ae7c-2d734992eb2c admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:05.967801 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abca56b-e50e-436e-ae7c-2d734992eb2c admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:05.973534 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d37182-6666-445e-ad90-835fc5b94ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:05.974149 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d37182-6666-445e-ad90-835fc5b94ad5 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:05.974422 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d37182-6666-445e-ad90-835fc5b94ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:05.974689 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d37182-6666-445e-ad90-835fc5b94ad5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:05.975324 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d37182-6666-445e-ad90-835fc5b94ad5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:05.977072 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d37182-6666-445e-ad90-835fc5b94ad5 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:05.991288 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 893/1772] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:21:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 00:21:05.993467 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08211b8c-646d-48cf-bcae-3b4aed504a94 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:06.005493 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08211b8c-646d-48cf-bcae-3b4aed504a94 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:06.010884 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbbfd31-72a6-4591-ba2c-24199fe6aac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:06.012361 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbbfd31-72a6-4591-ba2c-24199fe6aac8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:06.012842 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbbfd31-72a6-4591-ba2c-24199fe6aac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:06.013647 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbbfd31-72a6-4591-ba2c-24199fe6aac8 admin admin] PATH_INFO: `/v3/projects/419dc0c570054c0aa8772d187a0403ec/users/fd89033279e949c980af6596eccc85a0/roles/ef689f96442945e8a7bc8cbfee13d909` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:06.030415 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbbfd31-72a6-4591-ba2c-24199fe6aac8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=419dc0c570054c0aa8772d187a0403ec, user_id=fd89033279e949c980af6596eccc85a0, role_id=ef689f96442945e8a7bc8cbfee13d909)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:06.033113 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbbfd31-72a6-4591-ba2c-24199fe6aac8 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:06.157188 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 880/1773] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:21:05 2022] PUT /identity/v3/projects/419dc0c570054c0aa8772d187a0403ec/users/fd89033279e949c980af6596eccc85a0/roles/ef689f96442945e8a7bc8cbfee13d909 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:21:06.167518 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f05b830-81de-46f2-b4e3-6f3b882e0f09 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:06.168060 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f05b830-81de-46f2-b4e3-6f3b882e0f09 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:06.168536 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f05b830-81de-46f2-b4e3-6f3b882e0f09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:06.199427 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-4f05b830-81de-46f2-b4e3-6f3b882e0f09 None None] MFA Rules not processed for user `fd89033279e949c980af6596eccc85a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:21:06.200729 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-4f05b830-81de-46f2-b4e3-6f3b882e0f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:06.274598 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 894/1774] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:21:06 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 00:21:06.278960 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbbfd31-72a6-4591-ba2c-24199fe6aac8 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:06.348483 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbbfd31-72a6-4591-ba2c-24199fe6aac8 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:06.353862 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e27d7e8-2682-4e85-a8bd-fc7eac0f1ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:06.354482 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e27d7e8-2682-4e85-a8bd-fc7eac0f1ef1 admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:06.354798 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e27d7e8-2682-4e85-a8bd-fc7eac0f1ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:06.355075 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e27d7e8-2682-4e85-a8bd-fc7eac0f1ef1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:06.355828 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e27d7e8-2682-4e85-a8bd-fc7eac0f1ef1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:06.357774 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e27d7e8-2682-4e85-a8bd-fc7eac0f1ef1 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:06.535405 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 881/1775] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:21:06 2022] POST /identity/v3/projects => generated 385 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:21:06.552258 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f05b830-81de-46f2-b4e3-6f3b882e0f09 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:06.563644 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f05b830-81de-46f2-b4e3-6f3b882e0f09 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:06.621928 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2285b4e-6c6e-4875-bb47-5236d8eaf56f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:06.622540 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2285b4e-6c6e-4875-bb47-5236d8eaf56f admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:06.622880 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2285b4e-6c6e-4875-bb47-5236d8eaf56f admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:06.623150 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2285b4e-6c6e-4875-bb47-5236d8eaf56f admin admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:06.623826 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2285b4e-6c6e-4875-bb47-5236d8eaf56f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:06.625728 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2285b4e-6c6e-4875-bb47-5236d8eaf56f admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:06.849471 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e27d7e8-2682-4e85-a8bd-fc7eac0f1ef1 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:06.849471 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-7e27d7e8-2682-4e85-a8bd-fc7eac0f1ef1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:06.856242 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e27d7e8-2682-4e85-a8bd-fc7eac0f1ef1 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:06.860038 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00633a8d-cf06-48e9-9309-9714699f43f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:06.860645 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00633a8d-cf06-48e9-9309-9714699f43f0 service cinder] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:06.860921 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00633a8d-cf06-48e9-9309-9714699f43f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:06.861300 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00633a8d-cf06-48e9-9309-9714699f43f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:06.862916 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-00633a8d-cf06-48e9-9309-9714699f43f0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:06.927618 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00633a8d-cf06-48e9-9309-9714699f43f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:06.929127 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00633a8d-cf06-48e9-9309-9714699f43f0 service cinder] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:06.938572 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 882/1776] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:21:06 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:21:07.073599 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 895/1777] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:21:06 2022] POST /identity/v3/users => generated 293 bytes in 523 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:21:07.087760 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00633a8d-cf06-48e9-9309-9714699f43f0 service cinder] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:07.088061 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00633a8d-cf06-48e9-9309-9714699f43f0 service cinder] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:07.093117 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a9c298-6269-4ee4-8cfa-94ddb376635d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:07.093714 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a9c298-6269-4ee4-8cfa-94ddb376635d admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:07.093986 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a9c298-6269-4ee4-8cfa-94ddb376635d admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:07.094258 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a9c298-6269-4ee4-8cfa-94ddb376635d admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:07.094886 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a9c298-6269-4ee4-8cfa-94ddb376635d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:07.096505 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a9c298-6269-4ee4-8cfa-94ddb376635d admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:07.107088 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 883/1778] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:21:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:21:07.110836 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2285b4e-6c6e-4875-bb47-5236d8eaf56f admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:07.120805 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2285b4e-6c6e-4875-bb47-5236d8eaf56f admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:07.125380 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3c133f-fc16-44ef-a89a-23034d6c1716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:07.125978 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3c133f-fc16-44ef-a89a-23034d6c1716 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:07.126255 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3c133f-fc16-44ef-a89a-23034d6c1716 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:07.126526 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3c133f-fc16-44ef-a89a-23034d6c1716 admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:07.127163 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3c133f-fc16-44ef-a89a-23034d6c1716 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:07.128815 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3c133f-fc16-44ef-a89a-23034d6c1716 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:07.138958 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 896/1779] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:21:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:21:07.142961 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a9c298-6269-4ee4-8cfa-94ddb376635d admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:07.153487 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a9c298-6269-4ee4-8cfa-94ddb376635d admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:07.158230 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23676a80-72ae-47e2-be9d-a3a7a2994c56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:07.166677 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23676a80-72ae-47e2-be9d-a3a7a2994c56 admin admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:07.166677 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23676a80-72ae-47e2-be9d-a3a7a2994c56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:07.166677 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23676a80-72ae-47e2-be9d-a3a7a2994c56 admin admin] PATH_INFO: `/v3/projects/fdd38fafaf2148759e10988f3293b792/users/868b802df35840899845c222c73cd83b/roles/ef689f96442945e8a7bc8cbfee13d909` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:07.174340 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23676a80-72ae-47e2-be9d-a3a7a2994c56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdd38fafaf2148759e10988f3293b792, user_id=868b802df35840899845c222c73cd83b, role_id=ef689f96442945e8a7bc8cbfee13d909)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:07.177184 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23676a80-72ae-47e2-be9d-a3a7a2994c56 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:07.249944 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 884/1780] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:21:07 2022] PUT /identity/v3/projects/fdd38fafaf2148759e10988f3293b792/users/868b802df35840899845c222c73cd83b/roles/ef689f96442945e8a7bc8cbfee13d909 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:21:07.261554 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b0bd28-1e5b-4082-91ac-e7f0edbb967d None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:07.261841 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b0bd28-1e5b-4082-91ac-e7f0edbb967d None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:07.262133 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b0bd28-1e5b-4082-91ac-e7f0edbb967d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:07.294392 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-15b0bd28-1e5b-4082-91ac-e7f0edbb967d None None] MFA Rules not processed for user `868b802df35840899845c222c73cd83b`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:21:07.295641 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-15b0bd28-1e5b-4082-91ac-e7f0edbb967d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:07.363745 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 897/1781] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:21:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 00:21:07.368446 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23676a80-72ae-47e2-be9d-a3a7a2994c56 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:07.429377 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23676a80-72ae-47e2-be9d-a3a7a2994c56 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:07.434208 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945c7fa4-cf6d-4a81-91f7-63c24a85b7a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:07.434810 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c7fa4-cf6d-4a81-91f7-63c24a85b7a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:07.435108 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c7fa4-cf6d-4a81-91f7-63c24a85b7a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:07.435360 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c7fa4-cf6d-4a81-91f7-63c24a85b7a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:07.436007 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945c7fa4-cf6d-4a81-91f7-63c24a85b7a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:07.437945 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945c7fa4-cf6d-4a81-91f7-63c24a85b7a1 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:07.467057 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:07.499031 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b0bd28-1e5b-4082-91ac-e7f0edbb967d None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:07.500722 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-15b0bd28-1e5b-4082-91ac-e7f0edbb967d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:07.514979 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b0bd28-1e5b-4082-91ac-e7f0edbb967d None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:07.519875 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b670d2ed-0726-48b6-925a-867fd586b33e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:07.520509 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b670d2ed-0726-48b6-925a-867fd586b33e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:07.520785 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b670d2ed-0726-48b6-925a-867fd586b33e admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:07.521055 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b670d2ed-0726-48b6-925a-867fd586b33e admin admin] PATH_INFO: `/v3/users/04104188d2214586b1cd6b72a355d80f` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:07.522209 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b670d2ed-0726-48b6-925a-867fd586b33e admin admin] RBAC: Authorizing `identity:delete_user(user_id=04104188d2214586b1cd6b72a355d80f)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:07.524897 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b670d2ed-0726-48b6-925a-867fd586b33e admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:07.564745 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 885/1782] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:21:07 2022] POST /identity/v3/projects => generated 385 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:21:07.568278 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945c7fa4-cf6d-4a81-91f7-63c24a85b7a1 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:07.633559 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945c7fa4-cf6d-4a81-91f7-63c24a85b7a1 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:07.639916 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07967d5a-eee4-44a9-8b78-f692407e3840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:07.642510 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07967d5a-eee4-44a9-8b78-f692407e3840 admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:07.642510 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07967d5a-eee4-44a9-8b78-f692407e3840 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:07.642510 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07967d5a-eee4-44a9-8b78-f692407e3840 admin admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:07.642510 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07967d5a-eee4-44a9-8b78-f692407e3840 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:07.643639 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07967d5a-eee4-44a9-8b78-f692407e3840 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:07.796419 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:07.816312 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-b670d2ed-0726-48b6-925a-867fd586b33e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04104188d2214586b1cd6b72a355d80f'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:07.817259 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-b670d2ed-0726-48b6-925a-867fd586b33e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04104188d2214586b1cd6b72a355d80f'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:07.839652 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-b670d2ed-0726-48b6-925a-867fd586b33e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04104188d2214586b1cd6b72a355d80f'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:07.872546 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 886/1783] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:21:07 2022] POST /identity/v3/users => generated 299 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 00:21:07.875303 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07967d5a-eee4-44a9-8b78-f692407e3840 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:07.877544 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-07967d5a-eee4-44a9-8b78-f692407e3840 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:07.890995 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07967d5a-eee4-44a9-8b78-f692407e3840 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:07.896257 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cf4b0c-4afa-404a-a16d-f4cb274aabbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:07.897019 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cf4b0c-4afa-404a-a16d-f4cb274aabbf admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:07.897302 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cf4b0c-4afa-404a-a16d-f4cb274aabbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:07.897572 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cf4b0c-4afa-404a-a16d-f4cb274aabbf admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:07.898363 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cf4b0c-4afa-404a-a16d-f4cb274aabbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:07.900005 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cf4b0c-4afa-404a-a16d-f4cb274aabbf admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:07.915231 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 887/1784] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:21:07 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:21:07.916347 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cf4b0c-4afa-404a-a16d-f4cb274aabbf admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:07.927080 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cf4b0c-4afa-404a-a16d-f4cb274aabbf admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:07.932227 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920c17ed-7dfd-4a22-92d9-5b0dad6548d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:07.935308 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c17ed-7dfd-4a22-92d9-5b0dad6548d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:07.935308 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c17ed-7dfd-4a22-92d9-5b0dad6548d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:07.935308 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c17ed-7dfd-4a22-92d9-5b0dad6548d1 admin admin] PATH_INFO: `/v3/projects/84532b956aaf4d3a9d3a022d5c1e7e66/users/d9be653da6c1438dbb8dc202339a16af/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:07.949360 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920c17ed-7dfd-4a22-92d9-5b0dad6548d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84532b956aaf4d3a9d3a022d5c1e7e66, user_id=d9be653da6c1438dbb8dc202339a16af, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:07.952175 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920c17ed-7dfd-4a22-92d9-5b0dad6548d1 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:07.982057 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-b670d2ed-0726-48b6-925a-867fd586b33e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04104188d2214586b1cd6b72a355d80f'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:07.982957 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-b670d2ed-0726-48b6-925a-867fd586b33e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04104188d2214586b1cd6b72a355d80f'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:07.998150 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 898/1785] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:21:07 2022] DELETE /identity/v3/users/04104188d2214586b1cd6b72a355d80f => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:21:08.002318 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a03d96f-ff6d-478f-ab9f-08bd8eb497ad None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:08.002661 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a03d96f-ff6d-478f-ab9f-08bd8eb497ad None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:08.002863 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a03d96f-ff6d-478f-ab9f-08bd8eb497ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:08.022714 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-8a03d96f-ff6d-478f-ab9f-08bd8eb497ad None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:21:08.023953 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-8a03d96f-ff6d-478f-ab9f-08bd8eb497ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:08.076377 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 888/1786] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:21:07 2022] PUT /identity/v3/projects/84532b956aaf4d3a9d3a022d5c1e7e66/users/d9be653da6c1438dbb8dc202339a16af/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:21:08.080202 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920c17ed-7dfd-4a22-92d9-5b0dad6548d1 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:08.089729 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 899/1787] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:21:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:21:08.099805 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a03d96f-ff6d-478f-ab9f-08bd8eb497ad None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:08.101252 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-8a03d96f-ff6d-478f-ab9f-08bd8eb497ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:08.143161 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920c17ed-7dfd-4a22-92d9-5b0dad6548d1 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:08.147867 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d310af-f577-40af-9495-7857b10cecfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:08.148477 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d310af-f577-40af-9495-7857b10cecfd admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:08.148756 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d310af-f577-40af-9495-7857b10cecfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:08.149026 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d310af-f577-40af-9495-7857b10cecfd admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:08.149662 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d310af-f577-40af-9495-7857b10cecfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:08.151303 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d310af-f577-40af-9495-7857b10cecfd admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:08.159424 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a03d96f-ff6d-478f-ab9f-08bd8eb497ad None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:08.164576 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 889/1788] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:21:08 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 00:21:08.176024 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d310af-f577-40af-9495-7857b10cecfd admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:08.193988 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d310af-f577-40af-9495-7857b10cecfd admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:08.204906 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbe815f-fef2-4b22-b6ac-6e1a2d72120a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:08.205805 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe815f-fef2-4b22-b6ac-6e1a2d72120a admin admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:08.209641 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe815f-fef2-4b22-b6ac-6e1a2d72120a admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:08.209641 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe815f-fef2-4b22-b6ac-6e1a2d72120a admin admin] PATH_INFO: `/v3/projects/84532b956aaf4d3a9d3a022d5c1e7e66/users/d9be653da6c1438dbb8dc202339a16af/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:08.211151 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbe815f-fef2-4b22-b6ac-6e1a2d72120a admin admin] RBAC: Authorizing `identity:create_grant(project_id=84532b956aaf4d3a9d3a022d5c1e7e66, user_id=d9be653da6c1438dbb8dc202339a16af, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:08.214401 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbe815f-fef2-4b22-b6ac-6e1a2d72120a admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:08.230077 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd32b79f-f8f0-49a6-a203-0119ab3ed3fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:08.231027 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd32b79f-f8f0-49a6-a203-0119ab3ed3fc service neutron] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:08.231844 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd32b79f-f8f0-49a6-a203-0119ab3ed3fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:08.231965 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd32b79f-f8f0-49a6-a203-0119ab3ed3fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:08.232285 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 890/1789] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:21:08 2022] PUT /identity/v3/projects/84532b956aaf4d3a9d3a022d5c1e7e66/users/d9be653da6c1438dbb8dc202339a16af/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:21:08.236593 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb0d04-304b-48ad-a939-84689330e7fb None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:08.236886 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb0d04-304b-48ad-a939-84689330e7fb None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:08.237800 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb0d04-304b-48ad-a939-84689330e7fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:08.243849 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd32b79f-f8f0-49a6-a203-0119ab3ed3fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:08.246142 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd32b79f-f8f0-49a6-a203-0119ab3ed3fc service neutron] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:08.257805 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 900/1790] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:21:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 00:21:08.270410 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-edcb0d04-304b-48ad-a939-84689330e7fb None None] MFA Rules not processed for user `d9be653da6c1438dbb8dc202339a16af`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:21:08.271852 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-edcb0d04-304b-48ad-a939-84689330e7fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:08.366754 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 891/1791] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:21:08 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 00:21:08.378816 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd32b79f-f8f0-49a6-a203-0119ab3ed3fc service neutron] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:08.380297 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-dd32b79f-f8f0-49a6-a203-0119ab3ed3fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:08.437430 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd32b79f-f8f0-49a6-a203-0119ab3ed3fc service neutron] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:08.489820 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e4ad35-9371-4cb1-9f0b-f5c72162afee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:08.490469 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e4ad35-9371-4cb1-9f0b-f5c72162afee service cinder] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:08.490941 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e4ad35-9371-4cb1-9f0b-f5c72162afee service cinder] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:08.491135 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e4ad35-9371-4cb1-9f0b-f5c72162afee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:08.493077 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-93e4ad35-9371-4cb1-9f0b-f5c72162afee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:08.558430 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e4ad35-9371-4cb1-9f0b-f5c72162afee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:08.560054 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e4ad35-9371-4cb1-9f0b-f5c72162afee service cinder] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:08.570613 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 901/1792] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:21:08 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:21:08.711083 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcb0d04-304b-48ad-a939-84689330e7fb None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:08.712748 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-edcb0d04-304b-48ad-a939-84689330e7fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:08.779386 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcb0d04-304b-48ad-a939-84689330e7fb None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:08.783955 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e66641-e75b-4aa6-9ae8-c596399952f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:08.784669 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e66641-e75b-4aa6-9ae8-c596399952f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:08.784866 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e66641-e75b-4aa6-9ae8-c596399952f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:08.785308 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e66641-e75b-4aa6-9ae8-c596399952f1 admin admin] PATH_INFO: `/v3/projects/28b2469901dd4280aa12effd7dfb3490` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:08.786696 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e66641-e75b-4aa6-9ae8-c596399952f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28b2469901dd4280aa12effd7dfb3490)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:08.792743 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e66641-e75b-4aa6-9ae8-c596399952f1 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:08.935484 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-74e66641-e75b-4aa6-9ae8-c596399952f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28b2469901dd4280aa12effd7dfb3490', 'request_id': 'req-74e66641-e75b-4aa6-9ae8-c596399952f1'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:08.942004 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-74e66641-e75b-4aa6-9ae8-c596399952f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28b2469901dd4280aa12effd7dfb3490', 'request_id': 'req-74e66641-e75b-4aa6-9ae8-c596399952f1'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:08.977626 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:09.032369 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-74e66641-e75b-4aa6-9ae8-c596399952f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28b2469901dd4280aa12effd7dfb3490', 'request_id': 'req-74e66641-e75b-4aa6-9ae8-c596399952f1'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:09.044882 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-74e66641-e75b-4aa6-9ae8-c596399952f1 admin admin] The token cache is being invalidate because project 28b2469901dd4280aa12effd7dfb3490 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81746) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:21:09.045497 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-74e66641-e75b-4aa6-9ae8-c596399952f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:09.046173 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-74e66641-e75b-4aa6-9ae8-c596399952f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:09.048031 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 892/1793] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:21:08 2022] DELETE /identity/v3/projects/28b2469901dd4280aa12effd7dfb3490 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:21:09.051952 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e4ad35-9371-4cb1-9f0b-f5c72162afee service cinder] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:09.064753 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e4ad35-9371-4cb1-9f0b-f5c72162afee service cinder] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:09.128052 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd80733-b7d3-4957-bce3-014e4b80d456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:09.128784 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd80733-b7d3-4957-bce3-014e4b80d456 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:09.129146 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd80733-b7d3-4957-bce3-014e4b80d456 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:09.129422 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd80733-b7d3-4957-bce3-014e4b80d456 admin admin] PATH_INFO: `/v3/users/0a13cb34deaf43ffbd1a2c3378feef70` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:09.130681 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd80733-b7d3-4957-bce3-014e4b80d456 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a13cb34deaf43ffbd1a2c3378feef70)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:09.133209 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd80733-b7d3-4957-bce3-014e4b80d456 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:09.351426 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-bdd80733-b7d3-4957-bce3-014e4b80d456 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a13cb34deaf43ffbd1a2c3378feef70'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:09.352233 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-bdd80733-b7d3-4957-bce3-014e4b80d456 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a13cb34deaf43ffbd1a2c3378feef70'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:09.374074 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-bdd80733-b7d3-4957-bce3-014e4b80d456 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a13cb34deaf43ffbd1a2c3378feef70'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:09.390151 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:09.484496 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-bdd80733-b7d3-4957-bce3-014e4b80d456 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a13cb34deaf43ffbd1a2c3378feef70'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:09.484496 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-bdd80733-b7d3-4957-bce3-014e4b80d456 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a13cb34deaf43ffbd1a2c3378feef70'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:09.498270 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 902/1794] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:21:09 2022] DELETE /identity/v3/users/0a13cb34deaf43ffbd1a2c3378feef70 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:21:09.962289 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e66641-e75b-4aa6-9ae8-c596399952f1 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:09.963837 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-74e66641-e75b-4aa6-9ae8-c596399952f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:09.979358 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e66641-e75b-4aa6-9ae8-c596399952f1 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:10.037120 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:10.046956 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3353621c-1ffb-4b94-b46a-639d3b82b665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:10.047705 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3353621c-1ffb-4b94-b46a-639d3b82b665 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:10.048052 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3353621c-1ffb-4b94-b46a-639d3b82b665 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:10.048382 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3353621c-1ffb-4b94-b46a-639d3b82b665 admin admin] PATH_INFO: `/v3/projects/ac3ce78eaddd422bb743324bc38babe8` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:10.049733 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3353621c-1ffb-4b94-b46a-639d3b82b665 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac3ce78eaddd422bb743324bc38babe8)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:10.051722 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3353621c-1ffb-4b94-b46a-639d3b82b665 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:10.162594 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-3353621c-1ffb-4b94-b46a-639d3b82b665 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac3ce78eaddd422bb743324bc38babe8', 'request_id': 'req-3353621c-1ffb-4b94-b46a-639d3b82b665'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:10.172208 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-3353621c-1ffb-4b94-b46a-639d3b82b665 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac3ce78eaddd422bb743324bc38babe8', 'request_id': 'req-3353621c-1ffb-4b94-b46a-639d3b82b665'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:10.249956 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-3353621c-1ffb-4b94-b46a-639d3b82b665 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac3ce78eaddd422bb743324bc38babe8', 'request_id': 'req-3353621c-1ffb-4b94-b46a-639d3b82b665'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:10.262160 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-3353621c-1ffb-4b94-b46a-639d3b82b665 admin admin] The token cache is being invalidate because project ac3ce78eaddd422bb743324bc38babe8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81746) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:21:10.262455 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-3353621c-1ffb-4b94-b46a-639d3b82b665 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:10.263460 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-3353621c-1ffb-4b94-b46a-639d3b82b665 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:10.265270 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 893/1795] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:21:09 2022] DELETE /identity/v3/projects/ac3ce78eaddd422bb743324bc38babe8 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:21:10.270217 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da2b47b-b15a-4020-addb-45c63f8ef8fb None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:10.271317 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da2b47b-b15a-4020-addb-45c63f8ef8fb None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:10.271800 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da2b47b-b15a-4020-addb-45c63f8ef8fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:10.292693 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-5da2b47b-b15a-4020-addb-45c63f8ef8fb None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:21:10.293954 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-5da2b47b-b15a-4020-addb-45c63f8ef8fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:10.379291 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 903/1796] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:21:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:21:10.387265 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e044bfe-1182-494e-ad24-a8fa7955b8d3 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:10.387710 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e044bfe-1182-494e-ad24-a8fa7955b8d3 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:10.388318 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e044bfe-1182-494e-ad24-a8fa7955b8d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:10.409226 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-2e044bfe-1182-494e-ad24-a8fa7955b8d3 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:21:10.410749 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-2e044bfe-1182-494e-ad24-a8fa7955b8d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:10.419936 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 894/1797] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:21:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:21:10.424078 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da2b47b-b15a-4020-addb-45c63f8ef8fb None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:10.434811 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da2b47b-b15a-4020-addb-45c63f8ef8fb None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:10.439425 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89b35a2-3a56-4516-baca-c6f8fd98b570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:10.440028 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b35a2-3a56-4516-baca-c6f8fd98b570 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:10.440305 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b35a2-3a56-4516-baca-c6f8fd98b570 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:10.440573 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b35a2-3a56-4516-baca-c6f8fd98b570 admin admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:10.441216 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89b35a2-3a56-4516-baca-c6f8fd98b570 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:21:10.441514 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89b35a2-3a56-4516-baca-c6f8fd98b570 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:10.443521 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89b35a2-3a56-4516-baca-c6f8fd98b570 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:10.456915 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 904/1798] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Oct 12 00:21:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:21:10.461143 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e044bfe-1182-494e-ad24-a8fa7955b8d3 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:10.470262 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e044bfe-1182-494e-ad24-a8fa7955b8d3 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:10.475526 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3aa508-c29a-4488-b41e-1021fe194806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:10.476131 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3aa508-c29a-4488-b41e-1021fe194806 admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:10.476412 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3aa508-c29a-4488-b41e-1021fe194806 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:10.476681 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3aa508-c29a-4488-b41e-1021fe194806 admin admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:10.477380 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3aa508-c29a-4488-b41e-1021fe194806 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:10.479316 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3aa508-c29a-4488-b41e-1021fe194806 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:10.594979 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 895/1799] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:21:10 2022] POST /identity/v3/projects => generated 377 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:21:10.599129 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89b35a2-3a56-4516-baca-c6f8fd98b570 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:10.609989 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89b35a2-3a56-4516-baca-c6f8fd98b570 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:10.668573 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a69ea8-b429-4225-8c3f-2db02eae0738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:10.669189 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a69ea8-b429-4225-8c3f-2db02eae0738 admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:10.669464 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a69ea8-b429-4225-8c3f-2db02eae0738 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:10.669739 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a69ea8-b429-4225-8c3f-2db02eae0738 admin admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:10.670395 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a69ea8-b429-4225-8c3f-2db02eae0738 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:10.672403 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a69ea8-b429-4225-8c3f-2db02eae0738 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:10.773737 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3aa508-c29a-4488-b41e-1021fe194806 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:10.774042 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 905/1800] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:21:10 2022] POST /identity/v3/users => generated 289 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 00:21:10.775497 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-4c3aa508-c29a-4488-b41e-1021fe194806 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:10.778309 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a69ea8-b429-4225-8c3f-2db02eae0738 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:10.799988 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a69ea8-b429-4225-8c3f-2db02eae0738 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:10.805077 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b17790d-4487-4eee-b438-c19ee598d56e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:10.805848 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b17790d-4487-4eee-b438-c19ee598d56e admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:10.806015 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b17790d-4487-4eee-b438-c19ee598d56e admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:10.806483 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b17790d-4487-4eee-b438-c19ee598d56e admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:10.807281 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b17790d-4487-4eee-b438-c19ee598d56e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:10.809234 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b17790d-4487-4eee-b438-c19ee598d56e admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:10.830160 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 906/1801] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:21:10 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 00:21:10.831428 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b17790d-4487-4eee-b438-c19ee598d56e admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:10.838461 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3aa508-c29a-4488-b41e-1021fe194806 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:10.842157 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b17790d-4487-4eee-b438-c19ee598d56e admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:10.842952 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0d6bbc-7a44-4da6-accd-f22cac0d69eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:10.843598 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0d6bbc-7a44-4da6-accd-f22cac0d69eb service cinder] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:10.843872 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0d6bbc-7a44-4da6-accd-f22cac0d69eb service cinder] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:10.844312 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0d6bbc-7a44-4da6-accd-f22cac0d69eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:10.846540 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-ed0d6bbc-7a44-4da6-accd-f22cac0d69eb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:10.847120 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301c36b8-fe78-423e-a7fa-e95cf365e6d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:10.847769 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301c36b8-fe78-423e-a7fa-e95cf365e6d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:10.848049 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301c36b8-fe78-423e-a7fa-e95cf365e6d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:10.848319 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301c36b8-fe78-423e-a7fa-e95cf365e6d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:10.848969 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301c36b8-fe78-423e-a7fa-e95cf365e6d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:10.850587 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301c36b8-fe78-423e-a7fa-e95cf365e6d3 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:10.863164 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 907/1802] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:21:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 00:21:10.867641 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301c36b8-fe78-423e-a7fa-e95cf365e6d3 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:10.881270 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301c36b8-fe78-423e-a7fa-e95cf365e6d3 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:10.886515 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b80eddc-5501-4e17-98a7-3e2361ce662b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:10.887171 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b80eddc-5501-4e17-98a7-3e2361ce662b admin admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:10.887478 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b80eddc-5501-4e17-98a7-3e2361ce662b admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:10.887750 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b80eddc-5501-4e17-98a7-3e2361ce662b admin admin] PATH_INFO: `/v3/projects/1c7d4f40101b4eca840f65021a7e61d1/users/d03e8d96371b4711a7a1846b9ba2bb63/roles/ef689f96442945e8a7bc8cbfee13d909` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:10.906135 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b80eddc-5501-4e17-98a7-3e2361ce662b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c7d4f40101b4eca840f65021a7e61d1, user_id=d03e8d96371b4711a7a1846b9ba2bb63, role_id=ef689f96442945e8a7bc8cbfee13d909)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:10.909052 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b80eddc-5501-4e17-98a7-3e2361ce662b admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:10.921426 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0d6bbc-7a44-4da6-accd-f22cac0d69eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:10.922860 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0d6bbc-7a44-4da6-accd-f22cac0d69eb service cinder] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:10.931848 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 896/1803] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:21:10 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:21:10.961310 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:10.991207 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 908/1804] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:21:10 2022] PUT /identity/v3/projects/1c7d4f40101b4eca840f65021a7e61d1/users/d03e8d96371b4711a7a1846b9ba2bb63/roles/ef689f96442945e8a7bc8cbfee13d909 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:21:11.002886 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fa2105-aede-41fe-8f2d-9f8d28f003e4 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:11.003467 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fa2105-aede-41fe-8f2d-9f8d28f003e4 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:11.003467 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fa2105-aede-41fe-8f2d-9f8d28f003e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:11.033434 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-01fa2105-aede-41fe-8f2d-9f8d28f003e4 None None] MFA Rules not processed for user `d03e8d96371b4711a7a1846b9ba2bb63`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:21:11.034582 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-01fa2105-aede-41fe-8f2d-9f8d28f003e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:11.103460 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 897/1805] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:21:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:21:11.107911 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b80eddc-5501-4e17-98a7-3e2361ce662b admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:11.172508 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b80eddc-5501-4e17-98a7-3e2361ce662b admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:11.177165 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389864a9-d06f-44fe-abe5-d085d279e609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:11.177774 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389864a9-d06f-44fe-abe5-d085d279e609 admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:11.178052 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389864a9-d06f-44fe-abe5-d085d279e609 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:11.178322 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389864a9-d06f-44fe-abe5-d085d279e609 admin admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:11.178944 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389864a9-d06f-44fe-abe5-d085d279e609 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:11.180876 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389864a9-d06f-44fe-abe5-d085d279e609 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:11.288995 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 909/1806] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:21:11 2022] POST /identity/v3/projects => generated 377 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:21:11.290200 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fa2105-aede-41fe-8f2d-9f8d28f003e4 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:11.301069 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fa2105-aede-41fe-8f2d-9f8d28f003e4 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:11.359105 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0041bc40-4f0d-46e2-a238-e2613ea87f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:11.359842 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0041bc40-4f0d-46e2-a238-e2613ea87f6d admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:11.360120 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0041bc40-4f0d-46e2-a238-e2613ea87f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:11.360397 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0041bc40-4f0d-46e2-a238-e2613ea87f6d admin admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:11.361041 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0041bc40-4f0d-46e2-a238-e2613ea87f6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:11.363064 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0041bc40-4f0d-46e2-a238-e2613ea87f6d admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:11.447815 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 898/1807] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:21:11 2022] POST /identity/v3/users => generated 289 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:21:11.452224 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389864a9-d06f-44fe-abe5-d085d279e609 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:11.462844 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389864a9-d06f-44fe-abe5-d085d279e609 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:11.469354 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27d2c9c-617d-42ff-af04-13081d9ffdc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:11.469966 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27d2c9c-617d-42ff-af04-13081d9ffdc2 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:11.470240 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27d2c9c-617d-42ff-af04-13081d9ffdc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:11.470510 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27d2c9c-617d-42ff-af04-13081d9ffdc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:11.471261 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27d2c9c-617d-42ff-af04-13081d9ffdc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:11.472877 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27d2c9c-617d-42ff-af04-13081d9ffdc2 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:11.486768 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 910/1808] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:21:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:21:11.487974 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0041bc40-4f0d-46e2-a238-e2613ea87f6d admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:11.501353 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0041bc40-4f0d-46e2-a238-e2613ea87f6d admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:11.506407 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bde14e-9614-477f-bb07-d110a701efd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:11.507018 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bde14e-9614-477f-bb07-d110a701efd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:11.507291 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bde14e-9614-477f-bb07-d110a701efd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:11.507748 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bde14e-9614-477f-bb07-d110a701efd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:11.508306 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bde14e-9614-477f-bb07-d110a701efd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:11.509896 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bde14e-9614-477f-bb07-d110a701efd4 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:11.524361 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 899/1809] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:21:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:21:11.525651 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27d2c9c-617d-42ff-af04-13081d9ffdc2 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:11.530216 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:11.536083 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27d2c9c-617d-42ff-af04-13081d9ffdc2 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:11.541496 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd8b6cb-c548-4880-ace4-c8ddf9abed93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:11.542155 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd8b6cb-c548-4880-ace4-c8ddf9abed93 admin admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:11.542434 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd8b6cb-c548-4880-ace4-c8ddf9abed93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:11.542706 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd8b6cb-c548-4880-ace4-c8ddf9abed93 admin admin] PATH_INFO: `/v3/projects/a191729619e347afb8c7da2babe458bf/users/7acc9c284d194ba89a7fa653a3fbb422/roles/ef689f96442945e8a7bc8cbfee13d909` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:11.564892 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd8b6cb-c548-4880-ace4-c8ddf9abed93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a191729619e347afb8c7da2babe458bf, user_id=7acc9c284d194ba89a7fa653a3fbb422, role_id=ef689f96442945e8a7bc8cbfee13d909)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:11.567687 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd8b6cb-c548-4880-ace4-c8ddf9abed93 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:11.680380 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 911/1810] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:21:11 2022] PUT /identity/v3/projects/a191729619e347afb8c7da2babe458bf/users/7acc9c284d194ba89a7fa653a3fbb422/roles/ef689f96442945e8a7bc8cbfee13d909 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:21:11.692294 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca74925-5689-406d-9e59-c66303bad412 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:11.692578 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca74925-5689-406d-9e59-c66303bad412 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:11.692851 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca74925-5689-406d-9e59-c66303bad412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:11.721731 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-0ca74925-5689-406d-9e59-c66303bad412 None None] MFA Rules not processed for user `7acc9c284d194ba89a7fa653a3fbb422`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:21:11.722953 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-0ca74925-5689-406d-9e59-c66303bad412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:11.764778 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:11.797717 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 900/1811] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:21:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:21:11.798543 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd8b6cb-c548-4880-ace4-c8ddf9abed93 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:11.868541 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd8b6cb-c548-4880-ace4-c8ddf9abed93 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:11.874912 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498a953f-97a6-4ead-b699-e5906213d795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:11.874912 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498a953f-97a6-4ead-b699-e5906213d795 admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:11.874912 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498a953f-97a6-4ead-b699-e5906213d795 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:11.875236 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498a953f-97a6-4ead-b699-e5906213d795 admin admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:11.875662 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498a953f-97a6-4ead-b699-e5906213d795 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:11.877745 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498a953f-97a6-4ead-b699-e5906213d795 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:11.989174 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 912/1812] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:21:11 2022] POST /identity/v3/projects => generated 377 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 00:21:11.993287 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca74925-5689-406d-9e59-c66303bad412 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:12.005124 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca74925-5689-406d-9e59-c66303bad412 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:12.071471 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022d8204-a599-4ab3-9425-3b14d619c536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:12.071471 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022d8204-a599-4ab3-9425-3b14d619c536 admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:12.071471 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022d8204-a599-4ab3-9425-3b14d619c536 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:12.071471 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022d8204-a599-4ab3-9425-3b14d619c536 admin admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:12.072045 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022d8204-a599-4ab3-9425-3b14d619c536 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:12.072045 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022d8204-a599-4ab3-9425-3b14d619c536 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:12.185337 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 901/1813] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:21:11 2022] POST /identity/v3/users => generated 295 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:21:12.189052 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498a953f-97a6-4ead-b699-e5906213d795 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:12.201234 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498a953f-97a6-4ead-b699-e5906213d795 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:12.206089 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599c5e1b-4c9a-407b-88dd-0ddbae24e841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:12.206857 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599c5e1b-4c9a-407b-88dd-0ddbae24e841 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:12.210995 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599c5e1b-4c9a-407b-88dd-0ddbae24e841 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:12.210995 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599c5e1b-4c9a-407b-88dd-0ddbae24e841 admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:12.210995 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599c5e1b-4c9a-407b-88dd-0ddbae24e841 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:12.210995 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599c5e1b-4c9a-407b-88dd-0ddbae24e841 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:12.221533 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 913/1814] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:21:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:21:12.225971 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022d8204-a599-4ab3-9425-3b14d619c536 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:12.237859 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022d8204-a599-4ab3-9425-3b14d619c536 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:12.242695 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67739be3-c4a4-41ae-8772-27c25c624084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:12.245672 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67739be3-c4a4-41ae-8772-27c25c624084 admin admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:12.245672 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67739be3-c4a4-41ae-8772-27c25c624084 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:12.245672 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67739be3-c4a4-41ae-8772-27c25c624084 admin admin] PATH_INFO: `/v3/projects/82538d41a63b49a282cba8ddaa19cdc0/users/9775b7c413ef4ab7b5c46a62b772df4d/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:12.260076 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67739be3-c4a4-41ae-8772-27c25c624084 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82538d41a63b49a282cba8ddaa19cdc0, user_id=9775b7c413ef4ab7b5c46a62b772df4d, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:12.262806 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67739be3-c4a4-41ae-8772-27c25c624084 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:12.356070 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 902/1815] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:21:12 2022] PUT /identity/v3/projects/82538d41a63b49a282cba8ddaa19cdc0/users/9775b7c413ef4ab7b5c46a62b772df4d/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 00:21:12.360538 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599c5e1b-4c9a-407b-88dd-0ddbae24e841 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:12.370640 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599c5e1b-4c9a-407b-88dd-0ddbae24e841 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:12.429280 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdd12dc-63c8-4ec1-afbd-d841101cc478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:12.429903 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd12dc-63c8-4ec1-afbd-d841101cc478 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:12.430259 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd12dc-63c8-4ec1-afbd-d841101cc478 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:12.430456 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd12dc-63c8-4ec1-afbd-d841101cc478 admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:12.431087 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdd12dc-63c8-4ec1-afbd-d841101cc478 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:12.432700 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdd12dc-63c8-4ec1-afbd-d841101cc478 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:12.442924 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 914/1816] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:21:12 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 00:21:12.444360 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:12.447557 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67739be3-c4a4-41ae-8772-27c25c624084 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:12.458337 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:12.461104 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67739be3-c4a4-41ae-8772-27c25c624084 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:12.469096 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17662e1-5c76-4409-8090-0c6773720ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:12.470268 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17662e1-5c76-4409-8090-0c6773720ad2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:12.470574 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17662e1-5c76-4409-8090-0c6773720ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:12.472046 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17662e1-5c76-4409-8090-0c6773720ad2 admin admin] PATH_INFO: `/v3/projects/82538d41a63b49a282cba8ddaa19cdc0/users/9775b7c413ef4ab7b5c46a62b772df4d/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:12.475506 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17662e1-5c76-4409-8090-0c6773720ad2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82538d41a63b49a282cba8ddaa19cdc0, user_id=9775b7c413ef4ab7b5c46a62b772df4d, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:12.482254 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17662e1-5c76-4409-8090-0c6773720ad2 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:12.527125 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 903/1817] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:21:12 2022] PUT /identity/v3/projects/82538d41a63b49a282cba8ddaa19cdc0/users/9775b7c413ef4ab7b5c46a62b772df4d/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 00:21:12.528315 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7fbf17-ebf1-4fd1-9d45-7f0ee867569a None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:12.528847 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7fbf17-ebf1-4fd1-9d45-7f0ee867569a None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:12.529328 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7fbf17-ebf1-4fd1-9d45-7f0ee867569a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:12.573011 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-dd7fbf17-ebf1-4fd1-9d45-7f0ee867569a None None] MFA Rules not processed for user `9775b7c413ef4ab7b5c46a62b772df4d`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:21:12.575270 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-dd7fbf17-ebf1-4fd1-9d45-7f0ee867569a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:12.612840 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17662e1-5c76-4409-8090-0c6773720ad2 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:12.614624 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-f17662e1-5c76-4409-8090-0c6773720ad2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:12.708746 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 915/1818] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:21:12 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:21:12.711006 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7fbf17-ebf1-4fd1-9d45-7f0ee867569a None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:12.731551 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17662e1-5c76-4409-8090-0c6773720ad2 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:12.736471 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3b65fe-ead5-4527-9f83-8cd39833e09a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:12.737112 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3b65fe-ead5-4527-9f83-8cd39833e09a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:12.737388 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3b65fe-ead5-4527-9f83-8cd39833e09a admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:12.737659 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3b65fe-ead5-4527-9f83-8cd39833e09a admin admin] PATH_INFO: `/v3/users/af2e7c0875e942eaa03113a2d5f9ea19` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:12.738807 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3b65fe-ead5-4527-9f83-8cd39833e09a admin admin] RBAC: Authorizing `identity:delete_user(user_id=af2e7c0875e942eaa03113a2d5f9ea19)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:12.740752 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3b65fe-ead5-4527-9f83-8cd39833e09a admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:12.791012 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7fbf17-ebf1-4fd1-9d45-7f0ee867569a None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:12.795020 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cabea00-9d1f-4188-9749-02dd63c26975 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:12.796507 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cabea00-9d1f-4188-9749-02dd63c26975 service cinder] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:12.796507 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cabea00-9d1f-4188-9749-02dd63c26975 service cinder] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:12.796692 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cabea00-9d1f-4188-9749-02dd63c26975 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:12.798553 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-7cabea00-9d1f-4188-9749-02dd63c26975 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:12.867890 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cabea00-9d1f-4188-9749-02dd63c26975 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:12.869327 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cabea00-9d1f-4188-9749-02dd63c26975 service cinder] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:12.880673 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 916/1819] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:21:12 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 00:21:12.881351 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cabea00-9d1f-4188-9749-02dd63c26975 service cinder] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:12.884439 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-7cabea00-9d1f-4188-9749-02dd63c26975 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:12.895614 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cabea00-9d1f-4188-9749-02dd63c26975 service cinder] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:12.899488 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5570928c-7bee-4dae-937f-72f88997ea86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:12.900108 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5570928c-7bee-4dae-937f-72f88997ea86 service cinder] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:12.900381 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5570928c-7bee-4dae-937f-72f88997ea86 service cinder] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:12.900648 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5570928c-7bee-4dae-937f-72f88997ea86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:12.910946 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5570928c-7bee-4dae-937f-72f88997ea86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:12.912454 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5570928c-7bee-4dae-937f-72f88997ea86 service cinder] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:12.923834 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 917/1820] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:21:12 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:21:12.947078 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-0b3b65fe-ead5-4527-9f83-8cd39833e09a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af2e7c0875e942eaa03113a2d5f9ea19'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:12.947922 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-0b3b65fe-ead5-4527-9f83-8cd39833e09a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af2e7c0875e942eaa03113a2d5f9ea19'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:12.961465 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-0b3b65fe-ead5-4527-9f83-8cd39833e09a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af2e7c0875e942eaa03113a2d5f9ea19'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:13.044500 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-0b3b65fe-ead5-4527-9f83-8cd39833e09a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af2e7c0875e942eaa03113a2d5f9ea19'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:13.048401 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:13.048537 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-0b3b65fe-ead5-4527-9f83-8cd39833e09a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af2e7c0875e942eaa03113a2d5f9ea19'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:13.063977 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 904/1821] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:21:12 2022] DELETE /identity/v3/users/af2e7c0875e942eaa03113a2d5f9ea19 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 00:21:13.068504 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1db59f3-2ff6-4a2e-8f35-0843c4f26827 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:13.068786 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1db59f3-2ff6-4a2e-8f35-0843c4f26827 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:13.069056 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1db59f3-2ff6-4a2e-8f35-0843c4f26827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:13.076519 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3b65fe-ead5-4527-9f83-8cd39833e09a admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:13.078148 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-0b3b65fe-ead5-4527-9f83-8cd39833e09a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:13.090466 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-a1db59f3-2ff6-4a2e-8f35-0843c4f26827 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:21:13.092325 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-a1db59f3-2ff6-4a2e-8f35-0843c4f26827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:13.173386 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3b65fe-ead5-4527-9f83-8cd39833e09a admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:13.185308 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c458875-28b2-48d8-ae87-e7bae636743b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:13.185932 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c458875-28b2-48d8-ae87-e7bae636743b service cinder] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:13.187101 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c458875-28b2-48d8-ae87-e7bae636743b service cinder] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:13.187708 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c458875-28b2-48d8-ae87-e7bae636743b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:13.191928 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-8c458875-28b2-48d8-ae87-e7bae636743b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:13.202078 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 918/1822] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:21:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 00:21:13.213095 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1db59f3-2ff6-4a2e-8f35-0843c4f26827 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:13.215970 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-a1db59f3-2ff6-4a2e-8f35-0843c4f26827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:13.278708 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c458875-28b2-48d8-ae87-e7bae636743b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:13.279692 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c458875-28b2-48d8-ae87-e7bae636743b service cinder] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:13.292625 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 905/1823] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:21:13 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:21:13.300644 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1db59f3-2ff6-4a2e-8f35-0843c4f26827 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:13.304823 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e55d51f-6272-4635-9678-c66f0012b48a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:13.305863 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e55d51f-6272-4635-9678-c66f0012b48a service neutron] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:13.306383 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e55d51f-6272-4635-9678-c66f0012b48a service neutron] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:13.306863 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e55d51f-6272-4635-9678-c66f0012b48a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:13.338475 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:13.339225 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e55d51f-6272-4635-9678-c66f0012b48a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:13.340771 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e55d51f-6272-4635-9678-c66f0012b48a service neutron] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:13.350978 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 919/1824] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:21:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 00:21:13.885254 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:14.109283 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c458875-28b2-48d8-ae87-e7bae636743b service cinder] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:14.110946 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-8c458875-28b2-48d8-ae87-e7bae636743b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:14.149736 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c458875-28b2-48d8-ae87-e7bae636743b service cinder] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:14.171359 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f1d4ff-fcf6-417e-a0ba-af59e7ba53d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:14.178779 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f1d4ff-fcf6-417e-a0ba-af59e7ba53d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:14.178779 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f1d4ff-fcf6-417e-a0ba-af59e7ba53d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:14.178779 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f1d4ff-fcf6-417e-a0ba-af59e7ba53d5 admin admin] PATH_INFO: `/v3/projects/43a2f7ca74f54c289c1862ab5fb2163f` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:14.178779 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f1d4ff-fcf6-417e-a0ba-af59e7ba53d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43a2f7ca74f54c289c1862ab5fb2163f)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:14.191894 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f1d4ff-fcf6-417e-a0ba-af59e7ba53d5 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:14.381220 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-31f1d4ff-fcf6-417e-a0ba-af59e7ba53d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43a2f7ca74f54c289c1862ab5fb2163f', 'request_id': 'req-31f1d4ff-fcf6-417e-a0ba-af59e7ba53d5'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:14.399628 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-31f1d4ff-fcf6-417e-a0ba-af59e7ba53d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43a2f7ca74f54c289c1862ab5fb2163f', 'request_id': 'req-31f1d4ff-fcf6-417e-a0ba-af59e7ba53d5'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:14.464349 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:14.465872 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-31f1d4ff-fcf6-417e-a0ba-af59e7ba53d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43a2f7ca74f54c289c1862ab5fb2163f', 'request_id': 'req-31f1d4ff-fcf6-417e-a0ba-af59e7ba53d5'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:14.484170 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-31f1d4ff-fcf6-417e-a0ba-af59e7ba53d5 admin admin] The token cache is being invalidate because project 43a2f7ca74f54c289c1862ab5fb2163f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81746) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:21:14.484530 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-31f1d4ff-fcf6-417e-a0ba-af59e7ba53d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:14.486280 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-31f1d4ff-fcf6-417e-a0ba-af59e7ba53d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:14.488022 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 906/1825] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:21:14 2022] DELETE /identity/v3/projects/43a2f7ca74f54c289c1862ab5fb2163f => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:21:14.492019 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e55d51f-6272-4635-9678-c66f0012b48a service neutron] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:14.504451 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e55d51f-6272-4635-9678-c66f0012b48a service neutron] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:14.597277 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd15e221-1016-42c0-8c96-2d530aedbe17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:14.597919 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd15e221-1016-42c0-8c96-2d530aedbe17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:14.598197 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd15e221-1016-42c0-8c96-2d530aedbe17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:14.598490 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd15e221-1016-42c0-8c96-2d530aedbe17 admin admin] PATH_INFO: `/v3/users/90457bdd8d8f4c39a57e190675e9ad65` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:14.599653 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd15e221-1016-42c0-8c96-2d530aedbe17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90457bdd8d8f4c39a57e190675e9ad65)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:14.601601 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd15e221-1016-42c0-8c96-2d530aedbe17 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:14.862316 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-bd15e221-1016-42c0-8c96-2d530aedbe17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90457bdd8d8f4c39a57e190675e9ad65'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:14.863025 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-bd15e221-1016-42c0-8c96-2d530aedbe17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90457bdd8d8f4c39a57e190675e9ad65'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:14.875621 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-bd15e221-1016-42c0-8c96-2d530aedbe17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90457bdd8d8f4c39a57e190675e9ad65'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:14.995769 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-bd15e221-1016-42c0-8c96-2d530aedbe17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90457bdd8d8f4c39a57e190675e9ad65'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:14.997117 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-bd15e221-1016-42c0-8c96-2d530aedbe17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90457bdd8d8f4c39a57e190675e9ad65'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:15.018428 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 920/1826] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:21:14 2022] DELETE /identity/v3/users/90457bdd8d8f4c39a57e190675e9ad65 => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 00:21:15.351995 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:15.875890 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f1d4ff-fcf6-417e-a0ba-af59e7ba53d5 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:15.878558 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-31f1d4ff-fcf6-417e-a0ba-af59e7ba53d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:15.898413 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f1d4ff-fcf6-417e-a0ba-af59e7ba53d5 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:15.950757 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:15.988378 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beb2c2a-d169-45f2-8497-0fdd2f00aa9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:15.989042 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beb2c2a-d169-45f2-8497-0fdd2f00aa9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:15.989657 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beb2c2a-d169-45f2-8497-0fdd2f00aa9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:15.989948 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beb2c2a-d169-45f2-8497-0fdd2f00aa9a admin admin] PATH_INFO: `/v3/projects/a5d529eda9854769b44da851caed0ace` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:15.991061 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beb2c2a-d169-45f2-8497-0fdd2f00aa9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5d529eda9854769b44da851caed0ace)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:15.993069 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beb2c2a-d169-45f2-8497-0fdd2f00aa9a admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:16.186140 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-6beb2c2a-d169-45f2-8497-0fdd2f00aa9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5d529eda9854769b44da851caed0ace', 'request_id': 'req-6beb2c2a-d169-45f2-8497-0fdd2f00aa9a'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:16.193619 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-6beb2c2a-d169-45f2-8497-0fdd2f00aa9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5d529eda9854769b44da851caed0ace', 'request_id': 'req-6beb2c2a-d169-45f2-8497-0fdd2f00aa9a'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:16.261621 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-6beb2c2a-d169-45f2-8497-0fdd2f00aa9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5d529eda9854769b44da851caed0ace', 'request_id': 'req-6beb2c2a-d169-45f2-8497-0fdd2f00aa9a'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:16.272940 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-6beb2c2a-d169-45f2-8497-0fdd2f00aa9a admin admin] The token cache is being invalidate because project a5d529eda9854769b44da851caed0ace was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81746) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:21:16.273303 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-6beb2c2a-d169-45f2-8497-0fdd2f00aa9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:16.273996 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-6beb2c2a-d169-45f2-8497-0fdd2f00aa9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:16.275586 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 907/1827] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:21:15 2022] DELETE /identity/v3/projects/a5d529eda9854769b44da851caed0ace => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:21:16.279539 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd15e221-1016-42c0-8c96-2d530aedbe17 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:16.280941 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-bd15e221-1016-42c0-8c96-2d530aedbe17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:16.300213 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd15e221-1016-42c0-8c96-2d530aedbe17 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:16.380640 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb32d70a-83b7-4701-87db-b5838b55e5ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:16.381363 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb32d70a-83b7-4701-87db-b5838b55e5ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:16.381648 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb32d70a-83b7-4701-87db-b5838b55e5ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:16.384369 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb32d70a-83b7-4701-87db-b5838b55e5ec admin admin] PATH_INFO: `/v3/users/924bf25903a848b8a888f1f2ec9453dc` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:16.384369 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb32d70a-83b7-4701-87db-b5838b55e5ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=924bf25903a848b8a888f1f2ec9453dc)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:16.385568 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb32d70a-83b7-4701-87db-b5838b55e5ec admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:16.439347 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c107b8c5-36ff-4c76-9958-7b8ef21aebf3 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:16.439663 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c107b8c5-36ff-4c76-9958-7b8ef21aebf3 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:16.439938 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c107b8c5-36ff-4c76-9958-7b8ef21aebf3 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:16.441298 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 908/1828] 10.222.0.36 () {54 vars in 921 bytes} [Wed Oct 12 00:21:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 00:21:16.452300 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50cc5fc-3bda-42c6-8f27-d3c46d3bb95a None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:16.452582 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50cc5fc-3bda-42c6-8f27-d3c46d3bb95a None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:16.452855 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50cc5fc-3bda-42c6-8f27-d3c46d3bb95a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:16.477626 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-e50cc5fc-3bda-42c6-8f27-d3c46d3bb95a None None] MFA Rules not processed for user `1e528da752ed4c78bc747215909f6569`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:21:16.478844 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-e50cc5fc-3bda-42c6-8f27-d3c46d3bb95a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:16.573009 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 909/1829] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Oct 12 00:21:16 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 00:21:16.586779 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50cc5fc-3bda-42c6-8f27-d3c46d3bb95a None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:16.588229 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-e50cc5fc-3bda-42c6-8f27-d3c46d3bb95a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:16.628843 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-bb32d70a-83b7-4701-87db-b5838b55e5ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '924bf25903a848b8a888f1f2ec9453dc'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:16.631236 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50cc5fc-3bda-42c6-8f27-d3c46d3bb95a None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:16.631978 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-bb32d70a-83b7-4701-87db-b5838b55e5ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '924bf25903a848b8a888f1f2ec9453dc'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:16.646064 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee09de4-eb3c-445e-b96c-a434feb302fd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e528da752ed4c78bc747215909f6569', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:16.647950 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee09de4-eb3c-445e-b96c-a434feb302fd service placement] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:16.648366 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee09de4-eb3c-445e-b96c-a434feb302fd service placement] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:16.648724 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee09de4-eb3c-445e-b96c-a434feb302fd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:16.654340 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-9ee09de4-eb3c-445e-b96c-a434feb302fd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:16.665348 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-bb32d70a-83b7-4701-87db-b5838b55e5ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '924bf25903a848b8a888f1f2ec9453dc'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:16.677361 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee09de4-eb3c-445e-b96c-a434feb302fd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:16.679248 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee09de4-eb3c-445e-b96c-a434feb302fd service placement] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:16.691975 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 910/1830] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:21:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:21:16.692891 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee09de4-eb3c-445e-b96c-a434feb302fd service placement] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:16.694421 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-9ee09de4-eb3c-445e-b96c-a434feb302fd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:16.708194 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee09de4-eb3c-445e-b96c-a434feb302fd service placement] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:16.712794 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7e35d5-c0f2-45db-9c5d-ce56a824dbe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:16.713993 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7e35d5-c0f2-45db-9c5d-ce56a824dbe6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:16.714242 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7e35d5-c0f2-45db-9c5d-ce56a824dbe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:16.714766 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7e35d5-c0f2-45db-9c5d-ce56a824dbe6 admin admin] PATH_INFO: `/v3/users/b4dba90d485945e1821be4667d5a5bc9` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:16.715938 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7e35d5-c0f2-45db-9c5d-ce56a824dbe6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4dba90d485945e1821be4667d5a5bc9)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:16.717848 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7e35d5-c0f2-45db-9c5d-ce56a824dbe6 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:16.758847 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:16.806623 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-bb32d70a-83b7-4701-87db-b5838b55e5ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '924bf25903a848b8a888f1f2ec9453dc'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:16.807278 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-bb32d70a-83b7-4701-87db-b5838b55e5ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '924bf25903a848b8a888f1f2ec9453dc'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:16.819956 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 921/1831] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:21:16 2022] DELETE /identity/v3/users/924bf25903a848b8a888f1f2ec9453dc => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:21:16.964152 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-5c7e35d5-c0f2-45db-9c5d-ce56a824dbe6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4dba90d485945e1821be4667d5a5bc9'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:16.964811 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-5c7e35d5-c0f2-45db-9c5d-ce56a824dbe6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4dba90d485945e1821be4667d5a5bc9'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:16.976799 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-5c7e35d5-c0f2-45db-9c5d-ce56a824dbe6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4dba90d485945e1821be4667d5a5bc9'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:17.084457 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-5c7e35d5-c0f2-45db-9c5d-ce56a824dbe6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4dba90d485945e1821be4667d5a5bc9'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:17.087556 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-5c7e35d5-c0f2-45db-9c5d-ce56a824dbe6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4dba90d485945e1821be4667d5a5bc9'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:17.101176 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 911/1832] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:21:16 2022] DELETE /identity/v3/users/b4dba90d485945e1821be4667d5a5bc9 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:21:17.239529 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b3a146-5c3e-4e58-a84a-cc565cdf32cc None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:17.240182 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b3a146-5c3e-4e58-a84a-cc565cdf32cc None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:17.240749 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b3a146-5c3e-4e58-a84a-cc565cdf32cc None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:17.242438 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 922/1833] 10.222.0.36 () {54 vars in 921 bytes} [Wed Oct 12 00:21:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 00:21:17.251997 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe723269-df3d-457c-89fd-673d6e26bea6 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:17.252401 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe723269-df3d-457c-89fd-673d6e26bea6 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:17.252754 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe723269-df3d-457c-89fd-673d6e26bea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:17.275858 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-fe723269-df3d-457c-89fd-673d6e26bea6 None None] MFA Rules not processed for user `1e528da752ed4c78bc747215909f6569`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:21:17.277621 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-fe723269-df3d-457c-89fd-673d6e26bea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:17.325649 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b3a146-5c3e-4e58-a84a-cc565cdf32cc None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:17.328530 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-39b3a146-5c3e-4e58-a84a-cc565cdf32cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:17.350029 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 912/1834] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Oct 12 00:21:17 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:21:17.364729 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe723269-df3d-457c-89fd-673d6e26bea6 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:17.367530 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-fe723269-df3d-457c-89fd-673d6e26bea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:17.394529 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe723269-df3d-457c-89fd-673d6e26bea6 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:17.405337 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bb9654-2fbc-4741-aaa1-d545c5d19ec8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e528da752ed4c78bc747215909f6569', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:17.406030 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bb9654-2fbc-4741-aaa1-d545c5d19ec8 service placement] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:17.406384 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bb9654-2fbc-4741-aaa1-d545c5d19ec8 service placement] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:17.406721 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bb9654-2fbc-4741-aaa1-d545c5d19ec8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:17.420283 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bb9654-2fbc-4741-aaa1-d545c5d19ec8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:17.422161 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bb9654-2fbc-4741-aaa1-d545c5d19ec8 service placement] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:17.435436 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 913/1835] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:21:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:21:17.468128 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b3a146-5c3e-4e58-a84a-cc565cdf32cc None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:17.473962 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c8d219-fd04-4802-90e7-767deb808ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:17.475022 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c8d219-fd04-4802-90e7-767deb808ae6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:17.475527 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c8d219-fd04-4802-90e7-767deb808ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:17.476706 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c8d219-fd04-4802-90e7-767deb808ae6 admin admin] PATH_INFO: `/v3/projects/246401c287454d22b0dc653ed0a65525` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:17.478222 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c8d219-fd04-4802-90e7-767deb808ae6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=246401c287454d22b0dc653ed0a65525)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:17.480818 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c8d219-fd04-4802-90e7-767deb808ae6 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:17.487961 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bb9654-2fbc-4741-aaa1-d545c5d19ec8 service placement] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:17.489930 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-d4bb9654-2fbc-4741-aaa1-d545c5d19ec8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:17.512144 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bb9654-2fbc-4741-aaa1-d545c5d19ec8 service placement] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:17.522041 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5c0076-3df5-43b7-8c49-eae9b8bcdbae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:17.523265 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5c0076-3df5-43b7-8c49-eae9b8bcdbae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:17.523758 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5c0076-3df5-43b7-8c49-eae9b8bcdbae admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:17.524168 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5c0076-3df5-43b7-8c49-eae9b8bcdbae admin admin] PATH_INFO: `/v3/projects/bbfafa65b93241bcb62d41018f6ccd1f` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:17.525600 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5c0076-3df5-43b7-8c49-eae9b8bcdbae admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbfafa65b93241bcb62d41018f6ccd1f)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:17.530680 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5c0076-3df5-43b7-8c49-eae9b8bcdbae admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:17.667912 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-99c8d219-fd04-4802-90e7-767deb808ae6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '246401c287454d22b0dc653ed0a65525', 'request_id': 'req-99c8d219-fd04-4802-90e7-767deb808ae6'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:17.674391 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-99c8d219-fd04-4802-90e7-767deb808ae6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '246401c287454d22b0dc653ed0a65525', 'request_id': 'req-99c8d219-fd04-4802-90e7-767deb808ae6'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:17.692608 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-be5c0076-3df5-43b7-8c49-eae9b8bcdbae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbfafa65b93241bcb62d41018f6ccd1f', 'request_id': 'req-be5c0076-3df5-43b7-8c49-eae9b8bcdbae'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:17.700004 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-be5c0076-3df5-43b7-8c49-eae9b8bcdbae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbfafa65b93241bcb62d41018f6ccd1f', 'request_id': 'req-be5c0076-3df5-43b7-8c49-eae9b8bcdbae'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:17.770215 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:17.878798 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-99c8d219-fd04-4802-90e7-767deb808ae6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '246401c287454d22b0dc653ed0a65525', 'request_id': 'req-99c8d219-fd04-4802-90e7-767deb808ae6'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:17.891799 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-99c8d219-fd04-4802-90e7-767deb808ae6 admin admin] The token cache is being invalidate because project 246401c287454d22b0dc653ed0a65525 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81747) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:21:17.892035 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-99c8d219-fd04-4802-90e7-767deb808ae6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:17.892700 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-99c8d219-fd04-4802-90e7-767deb808ae6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:17.900175 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 923/1836] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:21:17 2022] DELETE /identity/v3/projects/246401c287454d22b0dc653ed0a65525 => generated 0 bytes in 570 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:21:17.902083 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e645b51-cb87-4bd7-84e5-f8e13ad430e3 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:17.902363 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e645b51-cb87-4bd7-84e5-f8e13ad430e3 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:17.902633 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e645b51-cb87-4bd7-84e5-f8e13ad430e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:17.926422 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-3e645b51-cb87-4bd7-84e5-f8e13ad430e3 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:21:17.928497 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-3e645b51-cb87-4bd7-84e5-f8e13ad430e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:17.970229 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-be5c0076-3df5-43b7-8c49-eae9b8bcdbae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbfafa65b93241bcb62d41018f6ccd1f', 'request_id': 'req-be5c0076-3df5-43b7-8c49-eae9b8bcdbae'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:17.986626 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-be5c0076-3df5-43b7-8c49-eae9b8bcdbae admin admin] The token cache is being invalidate because project bbfafa65b93241bcb62d41018f6ccd1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81746) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:21:17.987208 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-be5c0076-3df5-43b7-8c49-eae9b8bcdbae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:17.988112 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-be5c0076-3df5-43b7-8c49-eae9b8bcdbae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:17.990080 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 914/1837] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:21:17 2022] DELETE /identity/v3/projects/bbfafa65b93241bcb62d41018f6ccd1f => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:21:17.991013 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5c0076-3df5-43b7-8c49-eae9b8bcdbae admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:17.992416 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-be5c0076-3df5-43b7-8c49-eae9b8bcdbae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:18.022132 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 924/1838] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:21:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:21:18.023610 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b792743-cf91-4c0d-9f45-864e70340431 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:18.024022 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b792743-cf91-4c0d-9f45-864e70340431 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:18.024303 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b792743-cf91-4c0d-9f45-864e70340431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:18.046010 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-5b792743-cf91-4c0d-9f45-864e70340431 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:21:18.047071 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-5b792743-cf91-4c0d-9f45-864e70340431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:18.055623 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 925/1839] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:21:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:21:18.056363 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b792743-cf91-4c0d-9f45-864e70340431 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:18.058188 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-5b792743-cf91-4c0d-9f45-864e70340431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:18.071146 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b792743-cf91-4c0d-9f45-864e70340431 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:18.076708 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a9d970-b657-47b3-aa2f-ca6e29a13bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:18.077344 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5c0076-3df5-43b7-8c49-eae9b8bcdbae admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:18.077466 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a9d970-b657-47b3-aa2f-ca6e29a13bc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:18.077619 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a9d970-b657-47b3-aa2f-ca6e29a13bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:18.077893 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a9d970-b657-47b3-aa2f-ca6e29a13bc7 admin admin] PATH_INFO: `/v3/users/fd89033279e949c980af6596eccc85a0` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:18.079040 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a9d970-b657-47b3-aa2f-ca6e29a13bc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd89033279e949c980af6596eccc85a0)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:18.081062 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a9d970-b657-47b3-aa2f-ca6e29a13bc7 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:18.083099 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4219e90-a7a0-47c3-9e44-ee28d7a2a117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:18.084053 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4219e90-a7a0-47c3-9e44-ee28d7a2a117 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:18.084422 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4219e90-a7a0-47c3-9e44-ee28d7a2a117 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:18.084877 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4219e90-a7a0-47c3-9e44-ee28d7a2a117 admin admin] PATH_INFO: `/v3/users/d03e8d96371b4711a7a1846b9ba2bb63` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:18.086277 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4219e90-a7a0-47c3-9e44-ee28d7a2a117 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d03e8d96371b4711a7a1846b9ba2bb63)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:18.089005 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4219e90-a7a0-47c3-9e44-ee28d7a2a117 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:18.288296 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-30a9d970-b657-47b3-aa2f-ca6e29a13bc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd89033279e949c980af6596eccc85a0'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:18.289116 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-30a9d970-b657-47b3-aa2f-ca6e29a13bc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd89033279e949c980af6596eccc85a0'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:18.290906 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-c4219e90-a7a0-47c3-9e44-ee28d7a2a117 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd03e8d96371b4711a7a1846b9ba2bb63'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:18.292169 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-c4219e90-a7a0-47c3-9e44-ee28d7a2a117 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd03e8d96371b4711a7a1846b9ba2bb63'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:18.302429 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-30a9d970-b657-47b3-aa2f-ca6e29a13bc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd89033279e949c980af6596eccc85a0'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:18.304148 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-c4219e90-a7a0-47c3-9e44-ee28d7a2a117 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd03e8d96371b4711a7a1846b9ba2bb63'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:18.363137 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-30a9d970-b657-47b3-aa2f-ca6e29a13bc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd89033279e949c980af6596eccc85a0'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:18.363826 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-30a9d970-b657-47b3-aa2f-ca6e29a13bc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd89033279e949c980af6596eccc85a0'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:18.376480 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 926/1840] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:21:18 2022] DELETE /identity/v3/users/fd89033279e949c980af6596eccc85a0 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:21:18.377775 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcd07b7-1e01-468b-9577-470432ad8025 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:18.378053 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcd07b7-1e01-468b-9577-470432ad8025 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:18.378321 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcd07b7-1e01-468b-9577-470432ad8025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:18.397887 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-4bcd07b7-1e01-468b-9577-470432ad8025 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:21:18.399068 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-4bcd07b7-1e01-468b-9577-470432ad8025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:18.406284 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-c4219e90-a7a0-47c3-9e44-ee28d7a2a117 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd03e8d96371b4711a7a1846b9ba2bb63'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:18.406657 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:18.406912 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-c4219e90-a7a0-47c3-9e44-ee28d7a2a117 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd03e8d96371b4711a7a1846b9ba2bb63'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:18.420161 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 915/1841] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:21:17 2022] DELETE /identity/v3/users/d03e8d96371b4711a7a1846b9ba2bb63 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:21:18.421389 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2033f0ce-3717-42e9-ab9d-1b7ec8525801 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:18.421674 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2033f0ce-3717-42e9-ab9d-1b7ec8525801 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:18.421936 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2033f0ce-3717-42e9-ab9d-1b7ec8525801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:18.441351 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-2033f0ce-3717-42e9-ab9d-1b7ec8525801 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:21:18.442710 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-2033f0ce-3717-42e9-ab9d-1b7ec8525801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:18.462243 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 927/1842] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:21:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:21:18.463269 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcd07b7-1e01-468b-9577-470432ad8025 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:18.465091 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-4bcd07b7-1e01-468b-9577-470432ad8025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:18.476704 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcd07b7-1e01-468b-9577-470432ad8025 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:18.503952 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 916/1843] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:21:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:21:18.505214 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3248dd1-19f1-4043-acd2-89898aec28cb None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:18.505498 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3248dd1-19f1-4043-acd2-89898aec28cb None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:18.505782 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3248dd1-19f1-4043-acd2-89898aec28cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:18.511302 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:18.527335 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-d3248dd1-19f1-4043-acd2-89898aec28cb None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:21:18.528575 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-d3248dd1-19f1-4043-acd2-89898aec28cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:18.532679 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11910b8a-7485-450b-bcaa-2a9fa39d4f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:18.533432 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11910b8a-7485-450b-bcaa-2a9fa39d4f6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:18.533706 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11910b8a-7485-450b-bcaa-2a9fa39d4f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:18.533979 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11910b8a-7485-450b-bcaa-2a9fa39d4f6f admin admin] PATH_INFO: `/v3/users/a491001660e549db85a151e77711797c` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:18.535082 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11910b8a-7485-450b-bcaa-2a9fa39d4f6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a491001660e549db85a151e77711797c)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:18.539606 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 917/1844] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:21:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:21:18.539756 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11910b8a-7485-450b-bcaa-2a9fa39d4f6f admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:18.539848 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2901f4-7b63-4d60-b0f0-0cf315214ed8 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:18.539848 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2901f4-7b63-4d60-b0f0-0cf315214ed8 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:18.539848 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2901f4-7b63-4d60-b0f0-0cf315214ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:18.557725 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-1f2901f4-7b63-4d60-b0f0-0cf315214ed8 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:21:18.559018 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-1f2901f4-7b63-4d60-b0f0-0cf315214ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:18.566755 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 918/1845] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:21:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:21:18.567595 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2901f4-7b63-4d60-b0f0-0cf315214ed8 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:18.577912 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2901f4-7b63-4d60-b0f0-0cf315214ed8 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:18.582643 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee0c58c-efea-4eb1-b726-38b5b9ff4629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:18.583241 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee0c58c-efea-4eb1-b726-38b5b9ff4629 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:18.583560 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee0c58c-efea-4eb1-b726-38b5b9ff4629 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:18.587934 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee0c58c-efea-4eb1-b726-38b5b9ff4629 admin admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:18.587934 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee0c58c-efea-4eb1-b726-38b5b9ff4629 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:21:18.587934 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee0c58c-efea-4eb1-b726-38b5b9ff4629 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:18.587934 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee0c58c-efea-4eb1-b726-38b5b9ff4629 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:18.598526 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 919/1846] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Oct 12 00:21:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:21:18.599322 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee0c58c-efea-4eb1-b726-38b5b9ff4629 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:18.608692 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee0c58c-efea-4eb1-b726-38b5b9ff4629 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:18.613416 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3252ddc6-db19-4bba-bb24-ea980e09ace5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:18.613977 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3252ddc6-db19-4bba-bb24-ea980e09ace5 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:18.614255 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3252ddc6-db19-4bba-bb24-ea980e09ace5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:18.614529 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3252ddc6-db19-4bba-bb24-ea980e09ace5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:18.615136 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3252ddc6-db19-4bba-bb24-ea980e09ace5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:21:18.615472 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3252ddc6-db19-4bba-bb24-ea980e09ace5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:18.617033 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3252ddc6-db19-4bba-bb24-ea980e09ace5 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:18.628653 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 920/1847] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Oct 12 00:21:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:21:18.629401 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3252ddc6-db19-4bba-bb24-ea980e09ace5 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:18.633167 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-3252ddc6-db19-4bba-bb24-ea980e09ace5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:18.689079 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3252ddc6-db19-4bba-bb24-ea980e09ace5 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:18.692748 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9832f4-5982-435a-895b-4eeeee02bfc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:18.693356 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9832f4-5982-435a-895b-4eeeee02bfc0 service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:18.693637 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9832f4-5982-435a-895b-4eeeee02bfc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:18.693911 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9832f4-5982-435a-895b-4eeeee02bfc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:18.703771 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9832f4-5982-435a-895b-4eeeee02bfc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:18.705277 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9832f4-5982-435a-895b-4eeeee02bfc0 service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:18.715243 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 921/1848] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:21:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:21:18.716065 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9832f4-5982-435a-895b-4eeeee02bfc0 service neutron] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:18.717351 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-be9832f4-5982-435a-895b-4eeeee02bfc0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:18.727987 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9832f4-5982-435a-895b-4eeeee02bfc0 service neutron] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:18.731880 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b668a4af-5676-4b58-8c93-cc06ead04f00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:18.732482 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b668a4af-5676-4b58-8c93-cc06ead04f00 service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:18.732879 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b668a4af-5676-4b58-8c93-cc06ead04f00 service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:18.733411 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b668a4af-5676-4b58-8c93-cc06ead04f00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:18.743183 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b668a4af-5676-4b58-8c93-cc06ead04f00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:18.744713 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b668a4af-5676-4b58-8c93-cc06ead04f00 service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:18.756628 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 922/1849] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:21:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:21:18.757295 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b668a4af-5676-4b58-8c93-cc06ead04f00 service neutron] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:18.769842 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b668a4af-5676-4b58-8c93-cc06ead04f00 service neutron] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:18.778410 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-11910b8a-7485-450b-bcaa-2a9fa39d4f6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a491001660e549db85a151e77711797c'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:18.779407 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cba253f-2c1b-4f0b-b186-4756c46a26a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:18.780032 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cba253f-2c1b-4f0b-b186-4756c46a26a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:18.780332 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cba253f-2c1b-4f0b-b186-4756c46a26a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:18.780567 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cba253f-2c1b-4f0b-b186-4756c46a26a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:18.781207 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cba253f-2c1b-4f0b-b186-4756c46a26a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:18.783501 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-11910b8a-7485-450b-bcaa-2a9fa39d4f6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a491001660e549db85a151e77711797c'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:18.784836 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cba253f-2c1b-4f0b-b186-4756c46a26a0 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:18.800937 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-11910b8a-7485-450b-bcaa-2a9fa39d4f6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a491001660e549db85a151e77711797c'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:18.957883 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 923/1850] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:21:18 2022] POST /identity/v3/projects => generated 399 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:21:18.958935 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cba253f-2c1b-4f0b-b186-4756c46a26a0 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:18.966207 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-6cba253f-2c1b-4f0b-b186-4756c46a26a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:18.983945 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-11910b8a-7485-450b-bcaa-2a9fa39d4f6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a491001660e549db85a151e77711797c'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:18.984847 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-11910b8a-7485-450b-bcaa-2a9fa39d4f6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a491001660e549db85a151e77711797c'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:19.023622 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 928/1851] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:21:18 2022] DELETE /identity/v3/users/a491001660e549db85a151e77711797c => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:21:19.024542 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11910b8a-7485-450b-bcaa-2a9fa39d4f6f admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:19.025928 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-11910b8a-7485-450b-bcaa-2a9fa39d4f6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:19.044376 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cba253f-2c1b-4f0b-b186-4756c46a26a0 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:19.077542 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:19.124217 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6584b7e4-9ad5-474e-8386-7aad3c638446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:19.124691 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11910b8a-7485-450b-bcaa-2a9fa39d4f6f admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:19.125006 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6584b7e4-9ad5-474e-8386-7aad3c638446 admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:19.125492 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6584b7e4-9ad5-474e-8386-7aad3c638446 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:19.125963 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6584b7e4-9ad5-474e-8386-7aad3c638446 admin admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:19.126788 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6584b7e4-9ad5-474e-8386-7aad3c638446 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:19.129830 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6584b7e4-9ad5-474e-8386-7aad3c638446 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:19.139495 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91cf500-98a3-4e93-b732-13ca211d4cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:19.141048 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91cf500-98a3-4e93-b732-13ca211d4cb2 admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:19.141387 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91cf500-98a3-4e93-b732-13ca211d4cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:19.141726 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91cf500-98a3-4e93-b732-13ca211d4cb2 admin admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:19.143021 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91cf500-98a3-4e93-b732-13ca211d4cb2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:19.147842 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91cf500-98a3-4e93-b732-13ca211d4cb2 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:19.218492 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 924/1852] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:21:18 2022] POST /identity/v3/projects => generated 387 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:21:19.228655 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6584b7e4-9ad5-474e-8386-7aad3c638446 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:19.264028 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 929/1853] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:21:19 2022] POST /identity/v3/users => generated 300 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:21:19.267916 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91cf500-98a3-4e93-b732-13ca211d4cb2 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:19.287414 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91cf500-98a3-4e93-b732-13ca211d4cb2 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:19.315815 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6584b7e4-9ad5-474e-8386-7aad3c638446 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:19.323422 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065e7a88-bec1-4e71-a498-f1db14479945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:19.327401 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065e7a88-bec1-4e71-a498-f1db14479945 admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:19.327979 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065e7a88-bec1-4e71-a498-f1db14479945 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:19.328331 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065e7a88-bec1-4e71-a498-f1db14479945 admin admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:19.330055 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065e7a88-bec1-4e71-a498-f1db14479945 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:19.332278 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065e7a88-bec1-4e71-a498-f1db14479945 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:19.374097 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78d147e-1630-4280-b3ae-c0cb5c48dcfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:19.374810 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78d147e-1630-4280-b3ae-c0cb5c48dcfd admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:19.375085 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78d147e-1630-4280-b3ae-c0cb5c48dcfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:19.375350 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78d147e-1630-4280-b3ae-c0cb5c48dcfd admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:19.376115 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78d147e-1630-4280-b3ae-c0cb5c48dcfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:19.377660 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78d147e-1630-4280-b3ae-c0cb5c48dcfd admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:19.392083 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 930/1854] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:21:19 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:21:19.393531 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78d147e-1630-4280-b3ae-c0cb5c48dcfd admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:19.404509 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78d147e-1630-4280-b3ae-c0cb5c48dcfd admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:19.409505 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c08fd9b-d2a2-4ec9-9cfc-b3e7f9e5949b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:19.410213 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c08fd9b-d2a2-4ec9-9cfc-b3e7f9e5949b admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:19.410592 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c08fd9b-d2a2-4ec9-9cfc-b3e7f9e5949b admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:19.411058 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c08fd9b-d2a2-4ec9-9cfc-b3e7f9e5949b admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:19.411873 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c08fd9b-d2a2-4ec9-9cfc-b3e7f9e5949b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:19.413466 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c08fd9b-d2a2-4ec9-9cfc-b3e7f9e5949b admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:19.420399 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:19.426250 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 925/1855] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:21:19 2022] POST /identity/v3/users => generated 294 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:21:19.427052 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065e7a88-bec1-4e71-a498-f1db14479945 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:19.431132 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-065e7a88-bec1-4e71-a498-f1db14479945 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:19.437300 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 931/1856] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:21:19 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 00:21:19.437300 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c08fd9b-d2a2-4ec9-9cfc-b3e7f9e5949b admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:19.445807 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065e7a88-bec1-4e71-a498-f1db14479945 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:19.446758 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c08fd9b-d2a2-4ec9-9cfc-b3e7f9e5949b admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:19.450920 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c31024d-2872-4dbf-8ba7-871afded0054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:19.451554 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecaa882-5676-4142-9f70-ba880674d48e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:19.452055 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c31024d-2872-4dbf-8ba7-871afded0054 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:19.452194 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecaa882-5676-4142-9f70-ba880674d48e admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:19.452330 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c31024d-2872-4dbf-8ba7-871afded0054 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:19.452425 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecaa882-5676-4142-9f70-ba880674d48e admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:19.452598 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c31024d-2872-4dbf-8ba7-871afded0054 admin admin] PATH_INFO: `/v3/projects/063a92e4b88f43b88a05dcfda7e19d89` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:19.452688 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecaa882-5676-4142-9f70-ba880674d48e admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:19.453263 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecaa882-5676-4142-9f70-ba880674d48e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:19.453701 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c31024d-2872-4dbf-8ba7-871afded0054 admin admin] RBAC: Authorizing `identity:delete_project(project_id=063a92e4b88f43b88a05dcfda7e19d89)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:19.454836 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecaa882-5676-4142-9f70-ba880674d48e admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:19.455610 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c31024d-2872-4dbf-8ba7-871afded0054 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:19.465605 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 932/1857] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:21:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:21:19.466582 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecaa882-5676-4142-9f70-ba880674d48e admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:19.476101 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecaa882-5676-4142-9f70-ba880674d48e admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:19.481516 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4b17ed-d4a8-4d32-8da4-bcb2e47c841a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:19.482165 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4b17ed-d4a8-4d32-8da4-bcb2e47c841a admin admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:19.482448 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4b17ed-d4a8-4d32-8da4-bcb2e47c841a admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:19.483735 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4b17ed-d4a8-4d32-8da4-bcb2e47c841a admin admin] PATH_INFO: `/v3/projects/61a665795db4405db2f2602f7a6c1cfe/users/1bdc756abcc44428aaf1b2093cd9864d/roles/ef689f96442945e8a7bc8cbfee13d909` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:19.511965 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4b17ed-d4a8-4d32-8da4-bcb2e47c841a admin admin] RBAC: Authorizing `identity:create_grant(project_id=61a665795db4405db2f2602f7a6c1cfe, user_id=1bdc756abcc44428aaf1b2093cd9864d, role_id=ef689f96442945e8a7bc8cbfee13d909)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:19.516161 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4b17ed-d4a8-4d32-8da4-bcb2e47c841a admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:19.578180 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:19.612734 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 933/1858] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:21:19 2022] PUT /identity/v3/projects/61a665795db4405db2f2602f7a6c1cfe/users/1bdc756abcc44428aaf1b2093cd9864d/roles/ef689f96442945e8a7bc8cbfee13d909 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:21:19.613613 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4b17ed-d4a8-4d32-8da4-bcb2e47c841a admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:19.626118 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-1c31024d-2872-4dbf-8ba7-871afded0054 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '063a92e4b88f43b88a05dcfda7e19d89', 'request_id': 'req-1c31024d-2872-4dbf-8ba7-871afded0054'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:19.633009 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-1c31024d-2872-4dbf-8ba7-871afded0054 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '063a92e4b88f43b88a05dcfda7e19d89', 'request_id': 'req-1c31024d-2872-4dbf-8ba7-871afded0054'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:19.676054 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4b17ed-d4a8-4d32-8da4-bcb2e47c841a admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:19.703790 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:19.726161 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-1c31024d-2872-4dbf-8ba7-871afded0054 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '063a92e4b88f43b88a05dcfda7e19d89', 'request_id': 'req-1c31024d-2872-4dbf-8ba7-871afded0054'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:19.731601 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308714b6-8113-4dc8-b674-9a2d1646e9a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:19.732491 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308714b6-8113-4dc8-b674-9a2d1646e9a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:19.732769 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308714b6-8113-4dc8-b674-9a2d1646e9a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:19.733037 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308714b6-8113-4dc8-b674-9a2d1646e9a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:19.733825 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308714b6-8113-4dc8-b674-9a2d1646e9a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:19.735445 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308714b6-8113-4dc8-b674-9a2d1646e9a4 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:19.737838 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-1c31024d-2872-4dbf-8ba7-871afded0054 admin admin] The token cache is being invalidate because project 063a92e4b88f43b88a05dcfda7e19d89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81746) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:21:19.738128 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-1c31024d-2872-4dbf-8ba7-871afded0054 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:19.738802 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-1c31024d-2872-4dbf-8ba7-871afded0054 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:19.740350 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 926/1859] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:21:19 2022] DELETE /identity/v3/projects/063a92e4b88f43b88a05dcfda7e19d89 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:21:19.743955 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c31024d-2872-4dbf-8ba7-871afded0054 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:19.745276 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-1c31024d-2872-4dbf-8ba7-871afded0054 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:19.749267 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 934/1860] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:21:19 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:21:19.749267 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308714b6-8113-4dc8-b674-9a2d1646e9a4 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:19.749267 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-308714b6-8113-4dc8-b674-9a2d1646e9a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:19.758000 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c31024d-2872-4dbf-8ba7-871afded0054 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:19.760226 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308714b6-8113-4dc8-b674-9a2d1646e9a4 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:19.762741 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1e9144-9b8d-406b-a0b6-cfc9caf30203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:19.763675 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1e9144-9b8d-406b-a0b6-cfc9caf30203 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:19.763951 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1e9144-9b8d-406b-a0b6-cfc9caf30203 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:19.764223 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1e9144-9b8d-406b-a0b6-cfc9caf30203 admin admin] PATH_INFO: `/v3/projects/1c7d4f40101b4eca840f65021a7e61d1` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:19.765195 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748ffcf0-47c3-4fae-8e35-3bfcfc1ca6a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:19.765334 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1e9144-9b8d-406b-a0b6-cfc9caf30203 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c7d4f40101b4eca840f65021a7e61d1)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:19.765800 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ffcf0-47c3-4fae-8e35-3bfcfc1ca6a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:19.766071 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ffcf0-47c3-4fae-8e35-3bfcfc1ca6a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:19.766347 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ffcf0-47c3-4fae-8e35-3bfcfc1ca6a4 admin admin] PATH_INFO: `/v3/projects/419dc0c570054c0aa8772d187a0403ec` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:19.767220 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1e9144-9b8d-406b-a0b6-cfc9caf30203 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:19.767471 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748ffcf0-47c3-4fae-8e35-3bfcfc1ca6a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=419dc0c570054c0aa8772d187a0403ec)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:19.769638 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748ffcf0-47c3-4fae-8e35-3bfcfc1ca6a4 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:19.882745 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-5c1e9144-9b8d-406b-a0b6-cfc9caf30203 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c7d4f40101b4eca840f65021a7e61d1', 'request_id': 'req-5c1e9144-9b8d-406b-a0b6-cfc9caf30203'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:19.889214 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-5c1e9144-9b8d-406b-a0b6-cfc9caf30203 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c7d4f40101b4eca840f65021a7e61d1', 'request_id': 'req-5c1e9144-9b8d-406b-a0b6-cfc9caf30203'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:19.894319 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-748ffcf0-47c3-4fae-8e35-3bfcfc1ca6a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '419dc0c570054c0aa8772d187a0403ec', 'request_id': 'req-748ffcf0-47c3-4fae-8e35-3bfcfc1ca6a4'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:19.901456 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-748ffcf0-47c3-4fae-8e35-3bfcfc1ca6a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '419dc0c570054c0aa8772d187a0403ec', 'request_id': 'req-748ffcf0-47c3-4fae-8e35-3bfcfc1ca6a4'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:19.986565 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-5c1e9144-9b8d-406b-a0b6-cfc9caf30203 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c7d4f40101b4eca840f65021a7e61d1', 'request_id': 'req-5c1e9144-9b8d-406b-a0b6-cfc9caf30203'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:19.997598 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-5c1e9144-9b8d-406b-a0b6-cfc9caf30203 admin admin] The token cache is being invalidate because project 1c7d4f40101b4eca840f65021a7e61d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81746) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:21:19.997884 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-5c1e9144-9b8d-406b-a0b6-cfc9caf30203 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:19.998526 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-5c1e9144-9b8d-406b-a0b6-cfc9caf30203 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:20.000096 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 927/1861] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:21:19 2022] DELETE /identity/v3/projects/1c7d4f40101b4eca840f65021a7e61d1 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:21:20.001324 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8c7199-41f7-4a28-8887-88aa6fbea33b None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:20.001602 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8c7199-41f7-4a28-8887-88aa6fbea33b None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:20.001964 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8c7199-41f7-4a28-8887-88aa6fbea33b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:20.030073 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-0b8c7199-41f7-4a28-8887-88aa6fbea33b None None] MFA Rules not processed for user `1bdc756abcc44428aaf1b2093cd9864d`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:21:20.031277 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-0b8c7199-41f7-4a28-8887-88aa6fbea33b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:20.072060 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-748ffcf0-47c3-4fae-8e35-3bfcfc1ca6a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '419dc0c570054c0aa8772d187a0403ec', 'request_id': 'req-748ffcf0-47c3-4fae-8e35-3bfcfc1ca6a4'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:20.083389 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-748ffcf0-47c3-4fae-8e35-3bfcfc1ca6a4 admin admin] The token cache is being invalidate because project 419dc0c570054c0aa8772d187a0403ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81747) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:21:20.083697 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-748ffcf0-47c3-4fae-8e35-3bfcfc1ca6a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:20.084365 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-748ffcf0-47c3-4fae-8e35-3bfcfc1ca6a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:20.085879 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 935/1862] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:21:19 2022] DELETE /identity/v3/projects/419dc0c570054c0aa8772d187a0403ec => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:21:20.086632 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748ffcf0-47c3-4fae-8e35-3bfcfc1ca6a4 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:20.087928 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-748ffcf0-47c3-4fae-8e35-3bfcfc1ca6a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:20.094886 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 928/1863] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:21:20 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:21:20.096669 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8c7199-41f7-4a28-8887-88aa6fbea33b None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:20.098286 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-0b8c7199-41f7-4a28-8887-88aa6fbea33b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:20.155727 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748ffcf0-47c3-4fae-8e35-3bfcfc1ca6a4 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:20.160450 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6094fc96-78b2-4ad3-b81e-d05375261b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:20.161233 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6094fc96-78b2-4ad3-b81e-d05375261b1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:20.161532 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6094fc96-78b2-4ad3-b81e-d05375261b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:20.161813 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6094fc96-78b2-4ad3-b81e-d05375261b1c admin admin] PATH_INFO: `/v3/users/a2d0b932ba36433bb22f5eb15ad2c180` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:20.162950 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6094fc96-78b2-4ad3-b81e-d05375261b1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2d0b932ba36433bb22f5eb15ad2c180)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:20.164984 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6094fc96-78b2-4ad3-b81e-d05375261b1c admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:20.165423 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8c7199-41f7-4a28-8887-88aa6fbea33b None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:20.170009 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8780cb34-76de-439f-9631-c77c3f7c64dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:20.170663 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8780cb34-76de-439f-9631-c77c3f7c64dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:20.170990 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8780cb34-76de-439f-9631-c77c3f7c64dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:20.171297 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8780cb34-76de-439f-9631-c77c3f7c64dd admin admin] PATH_INFO: `/v3/projects/13bdb0f5c58a40af9e420f6072f2f2b0/users/63474c55edbf4b22ad609471a2414ede/roles/ef689f96442945e8a7bc8cbfee13d909` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:20.185530 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8780cb34-76de-439f-9631-c77c3f7c64dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=13bdb0f5c58a40af9e420f6072f2f2b0, user_id=63474c55edbf4b22ad609471a2414ede, role_id=ef689f96442945e8a7bc8cbfee13d909)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:20.188264 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8780cb34-76de-439f-9631-c77c3f7c64dd admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:20.206180 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:20.280857 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 929/1864] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:21:20 2022] PUT /identity/v3/projects/13bdb0f5c58a40af9e420f6072f2f2b0/users/63474c55edbf4b22ad609471a2414ede/roles/ef689f96442945e8a7bc8cbfee13d909 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:21:20.281721 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8780cb34-76de-439f-9631-c77c3f7c64dd admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:20.283327 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-8780cb34-76de-439f-9631-c77c3f7c64dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:20.342783 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-6094fc96-78b2-4ad3-b81e-d05375261b1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2d0b932ba36433bb22f5eb15ad2c180'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:20.343718 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-6094fc96-78b2-4ad3-b81e-d05375261b1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2d0b932ba36433bb22f5eb15ad2c180'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:20.349437 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8780cb34-76de-439f-9631-c77c3f7c64dd admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:20.356292 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69cbb96-fc16-4118-9174-4744df7de42a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:20.356292 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69cbb96-fc16-4118-9174-4744df7de42a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:20.356292 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69cbb96-fc16-4118-9174-4744df7de42a admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:20.356292 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69cbb96-fc16-4118-9174-4744df7de42a admin admin] PATH_INFO: `/v3/users/7acc9c284d194ba89a7fa653a3fbb422` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:20.356760 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69cbb96-fc16-4118-9174-4744df7de42a admin admin] RBAC: Authorizing `identity:delete_user(user_id=7acc9c284d194ba89a7fa653a3fbb422)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:20.358689 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69cbb96-fc16-4118-9174-4744df7de42a admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:20.359306 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-6094fc96-78b2-4ad3-b81e-d05375261b1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2d0b932ba36433bb22f5eb15ad2c180'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:20.460843 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-6094fc96-78b2-4ad3-b81e-d05375261b1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2d0b932ba36433bb22f5eb15ad2c180'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:20.461843 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-6094fc96-78b2-4ad3-b81e-d05375261b1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2d0b932ba36433bb22f5eb15ad2c180'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:20.474720 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 936/1865] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:21:20 2022] DELETE /identity/v3/users/a2d0b932ba36433bb22f5eb15ad2c180 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:21:20.475613 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6094fc96-78b2-4ad3-b81e-d05375261b1c admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:20.476932 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-6094fc96-78b2-4ad3-b81e-d05375261b1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:20.542608 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6094fc96-78b2-4ad3-b81e-d05375261b1c admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:20.547701 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5464cb-b88c-4e24-93ff-5ca6b6bbaf81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:20.548427 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5464cb-b88c-4e24-93ff-5ca6b6bbaf81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:20.548768 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5464cb-b88c-4e24-93ff-5ca6b6bbaf81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:20.549500 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5464cb-b88c-4e24-93ff-5ca6b6bbaf81 admin admin] PATH_INFO: `/v3/users/868b802df35840899845c222c73cd83b` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:20.550542 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5464cb-b88c-4e24-93ff-5ca6b6bbaf81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=868b802df35840899845c222c73cd83b)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:20.552652 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5464cb-b88c-4e24-93ff-5ca6b6bbaf81 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:20.582284 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-d69cbb96-fc16-4118-9174-4744df7de42a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7acc9c284d194ba89a7fa653a3fbb422'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:20.582946 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-d69cbb96-fc16-4118-9174-4744df7de42a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7acc9c284d194ba89a7fa653a3fbb422'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:20.595221 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-d69cbb96-fc16-4118-9174-4744df7de42a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7acc9c284d194ba89a7fa653a3fbb422'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:20.624075 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:20.677855 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-d69cbb96-fc16-4118-9174-4744df7de42a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7acc9c284d194ba89a7fa653a3fbb422'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:20.678491 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-d69cbb96-fc16-4118-9174-4744df7de42a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7acc9c284d194ba89a7fa653a3fbb422'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:20.691650 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 930/1866] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:21:20 2022] DELETE /identity/v3/users/7acc9c284d194ba89a7fa653a3fbb422 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:21:20.692367 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69cbb96-fc16-4118-9174-4744df7de42a admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:20.693734 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-d69cbb96-fc16-4118-9174-4744df7de42a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:20.750279 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69cbb96-fc16-4118-9174-4744df7de42a admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:20.753842 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535f558e-819b-4b0e-95be-f385818e71b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:20.754443 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f558e-819b-4b0e-95be-f385818e71b1 service cinder] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:20.754717 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f558e-819b-4b0e-95be-f385818e71b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:20.754984 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f558e-819b-4b0e-95be-f385818e71b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:20.756854 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-535f558e-819b-4b0e-95be-f385818e71b1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:20.769863 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:20.815800 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535f558e-819b-4b0e-95be-f385818e71b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:20.817241 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535f558e-819b-4b0e-95be-f385818e71b1 service cinder] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:20.837896 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 931/1867] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:21:20 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:21:20.840823 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3f88ca-b738-4526-a30a-b53fc483e706 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:20.841167 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3f88ca-b738-4526-a30a-b53fc483e706 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:20.841446 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3f88ca-b738-4526-a30a-b53fc483e706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:20.847763 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-0d5464cb-b88c-4e24-93ff-5ca6b6bbaf81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '868b802df35840899845c222c73cd83b'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:20.848541 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-0d5464cb-b88c-4e24-93ff-5ca6b6bbaf81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '868b802df35840899845c222c73cd83b'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:20.870190 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-9d3f88ca-b738-4526-a30a-b53fc483e706 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:21:20.871190 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-9d3f88ca-b738-4526-a30a-b53fc483e706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:20.881968 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-0d5464cb-b88c-4e24-93ff-5ca6b6bbaf81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '868b802df35840899845c222c73cd83b'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:20.932760 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 932/1868] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:21:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:21:20.933731 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3f88ca-b738-4526-a30a-b53fc483e706 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:20.935150 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-9d3f88ca-b738-4526-a30a-b53fc483e706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:20.947743 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3f88ca-b738-4526-a30a-b53fc483e706 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:20.952925 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707d31d1-9fc2-478c-a86f-4128ccd7dc94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:20.955704 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d31d1-9fc2-478c-a86f-4128ccd7dc94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:20.956214 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d31d1-9fc2-478c-a86f-4128ccd7dc94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:20.956608 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d31d1-9fc2-478c-a86f-4128ccd7dc94 admin admin] PATH_INFO: `/v3/projects/c809115d8a164a3992177fcd20ebc96a` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:20.958035 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707d31d1-9fc2-478c-a86f-4128ccd7dc94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c809115d8a164a3992177fcd20ebc96a)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:20.960402 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707d31d1-9fc2-478c-a86f-4128ccd7dc94 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:20.981121 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-0d5464cb-b88c-4e24-93ff-5ca6b6bbaf81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '868b802df35840899845c222c73cd83b'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:20.981757 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-0d5464cb-b88c-4e24-93ff-5ca6b6bbaf81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '868b802df35840899845c222c73cd83b'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:21.007012 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 937/1869] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:21:20 2022] DELETE /identity/v3/users/868b802df35840899845c222c73cd83b => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:21:21.008346 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5464cb-b88c-4e24-93ff-5ca6b6bbaf81 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:21.009847 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-0d5464cb-b88c-4e24-93ff-5ca6b6bbaf81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:21.036601 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:21.070081 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:21.084480 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-707d31d1-9fc2-478c-a86f-4128ccd7dc94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c809115d8a164a3992177fcd20ebc96a', 'request_id': 'req-707d31d1-9fc2-478c-a86f-4128ccd7dc94'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:21.086519 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5464cb-b88c-4e24-93ff-5ca6b6bbaf81 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:21.092770 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-707d31d1-9fc2-478c-a86f-4128ccd7dc94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c809115d8a164a3992177fcd20ebc96a', 'request_id': 'req-707d31d1-9fc2-478c-a86f-4128ccd7dc94'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:21.150366 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e516a17-89cd-4b0f-94e9-3e86bb18f229 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:21.150366 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e516a17-89cd-4b0f-94e9-3e86bb18f229 service neutron] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:21.150605 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e516a17-89cd-4b0f-94e9-3e86bb18f229 service neutron] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:21.151433 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e516a17-89cd-4b0f-94e9-3e86bb18f229 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:21.159738 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-707d31d1-9fc2-478c-a86f-4128ccd7dc94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c809115d8a164a3992177fcd20ebc96a', 'request_id': 'req-707d31d1-9fc2-478c-a86f-4128ccd7dc94'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:21.179982 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-707d31d1-9fc2-478c-a86f-4128ccd7dc94 admin admin] The token cache is being invalidate because project c809115d8a164a3992177fcd20ebc96a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81746) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:21:21.180519 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-707d31d1-9fc2-478c-a86f-4128ccd7dc94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:21.181340 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-707d31d1-9fc2-478c-a86f-4128ccd7dc94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:21.187970 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 933/1870] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:21:20 2022] DELETE /identity/v3/projects/c809115d8a164a3992177fcd20ebc96a => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:21:21.192429 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730f8c7d-09c6-4695-a3d1-c62186787342 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:21.193152 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730f8c7d-09c6-4695-a3d1-c62186787342 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:21.195052 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730f8c7d-09c6-4695-a3d1-c62186787342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:21.218711 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-730f8c7d-09c6-4695-a3d1-c62186787342 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:21:21.220316 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-730f8c7d-09c6-4695-a3d1-c62186787342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:21.238502 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e516a17-89cd-4b0f-94e9-3e86bb18f229 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:21.240116 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e516a17-89cd-4b0f-94e9-3e86bb18f229 service neutron] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:21.250399 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 938/1871] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:21:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:21:21.272391 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=88,keystone:INSERT=6,keystone:DELETE=6 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:21.285519 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 934/1872] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:21:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:21:21.295124 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d52f3f7-2e3d-4d08-b1a0-2c6b077180f5 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:21.295429 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d52f3f7-2e3d-4d08-b1a0-2c6b077180f5 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:21.295733 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d52f3f7-2e3d-4d08-b1a0-2c6b077180f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:21.314202 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730f8c7d-09c6-4695-a3d1-c62186787342 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:21.315659 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-730f8c7d-09c6-4695-a3d1-c62186787342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:21.322811 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-4d52f3f7-2e3d-4d08-b1a0-2c6b077180f5 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:21:21.324079 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-4d52f3f7-2e3d-4d08-b1a0-2c6b077180f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:21.331230 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730f8c7d-09c6-4695-a3d1-c62186787342 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:21.335242 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 939/1873] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:21:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:21:21.343202 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c66c01-7dd9-4356-916d-c6ee1006b9a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:21.343857 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c66c01-7dd9-4356-916d-c6ee1006b9a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:21.344134 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c66c01-7dd9-4356-916d-c6ee1006b9a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:21.345007 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c66c01-7dd9-4356-916d-c6ee1006b9a8 admin admin] PATH_INFO: `/v3/projects/a191729619e347afb8c7da2babe458bf` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:21.345707 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c66c01-7dd9-4356-916d-c6ee1006b9a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a191729619e347afb8c7da2babe458bf)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:21.347668 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c66c01-7dd9-4356-916d-c6ee1006b9a8 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:21.351557 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d52f3f7-2e3d-4d08-b1a0-2c6b077180f5 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:21.364520 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d52f3f7-2e3d-4d08-b1a0-2c6b077180f5 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:21.371142 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309d4656-35ea-406f-9597-a523e90fa095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:21.371997 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d4656-35ea-406f-9597-a523e90fa095 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:21.372411 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d4656-35ea-406f-9597-a523e90fa095 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:21.372855 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d4656-35ea-406f-9597-a523e90fa095 admin admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:21.373649 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309d4656-35ea-406f-9597-a523e90fa095 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:21:21.374101 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309d4656-35ea-406f-9597-a523e90fa095 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:21.375866 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309d4656-35ea-406f-9597-a523e90fa095 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:21.398014 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 940/1874] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Oct 12 00:21:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:21:21.398014 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309d4656-35ea-406f-9597-a523e90fa095 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:21.410169 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309d4656-35ea-406f-9597-a523e90fa095 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:21.415949 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3d9100-45e5-42c7-a2d3-ccff3b298dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:21.420866 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d9100-45e5-42c7-a2d3-ccff3b298dcd admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:21.420866 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d9100-45e5-42c7-a2d3-ccff3b298dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:21.420866 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d9100-45e5-42c7-a2d3-ccff3b298dcd admin admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:21.420866 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3d9100-45e5-42c7-a2d3-ccff3b298dcd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:21.420866 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3d9100-45e5-42c7-a2d3-ccff3b298dcd admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:21.496691 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-74c66c01-7dd9-4356-916d-c6ee1006b9a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a191729619e347afb8c7da2babe458bf', 'request_id': 'req-74c66c01-7dd9-4356-916d-c6ee1006b9a8'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:21.504155 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-74c66c01-7dd9-4356-916d-c6ee1006b9a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a191729619e347afb8c7da2babe458bf', 'request_id': 'req-74c66c01-7dd9-4356-916d-c6ee1006b9a8'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:21.534858 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 941/1875] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:21:21 2022] POST /identity/v3/projects => generated 385 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 00:21:21.539065 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3d9100-45e5-42c7-a2d3-ccff3b298dcd admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:21.604255 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3d9100-45e5-42c7-a2d3-ccff3b298dcd admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:21.610283 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45118592-abce-431e-922f-ec026f1b7cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:21.611801 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45118592-abce-431e-922f-ec026f1b7cfb admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:21.612094 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45118592-abce-431e-922f-ec026f1b7cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:21.612374 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45118592-abce-431e-922f-ec026f1b7cfb admin admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:21.613023 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45118592-abce-431e-922f-ec026f1b7cfb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:21.615001 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45118592-abce-431e-922f-ec026f1b7cfb admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:21.641224 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-74c66c01-7dd9-4356-916d-c6ee1006b9a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a191729619e347afb8c7da2babe458bf', 'request_id': 'req-74c66c01-7dd9-4356-916d-c6ee1006b9a8'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:21.663874 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-74c66c01-7dd9-4356-916d-c6ee1006b9a8 admin admin] The token cache is being invalidate because project a191729619e347afb8c7da2babe458bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81746) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:21:21.664175 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-74c66c01-7dd9-4356-916d-c6ee1006b9a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:21.665410 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-74c66c01-7dd9-4356-916d-c6ee1006b9a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:21.667493 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 935/1876] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:21:21 2022] DELETE /identity/v3/projects/a191729619e347afb8c7da2babe458bf => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 00:21:21.670962 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c66c01-7dd9-4356-916d-c6ee1006b9a8 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:21.672591 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-74c66c01-7dd9-4356-916d-c6ee1006b9a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:21.741837 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c66c01-7dd9-4356-916d-c6ee1006b9a8 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:21.746535 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fb12ae-f33a-40b8-8ce2-ba0948fd7235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:21.747265 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fb12ae-f33a-40b8-8ce2-ba0948fd7235 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:21.747585 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fb12ae-f33a-40b8-8ce2-ba0948fd7235 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:21.747857 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fb12ae-f33a-40b8-8ce2-ba0948fd7235 admin admin] PATH_INFO: `/v3/projects/fdd38fafaf2148759e10988f3293b792` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:21.748993 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fb12ae-f33a-40b8-8ce2-ba0948fd7235 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdd38fafaf2148759e10988f3293b792)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:21.750900 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fb12ae-f33a-40b8-8ce2-ba0948fd7235 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:21.781809 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 942/1877] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:21:21 2022] POST /identity/v3/users => generated 293 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:21:21.781809 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45118592-abce-431e-922f-ec026f1b7cfb admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:21.781809 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-45118592-abce-431e-922f-ec026f1b7cfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:21.793420 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45118592-abce-431e-922f-ec026f1b7cfb admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:21.801255 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05888c3b-d5fd-44d3-9af8-32c6533205f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:21.801255 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05888c3b-d5fd-44d3-9af8-32c6533205f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:21.801255 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05888c3b-d5fd-44d3-9af8-32c6533205f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:21.801255 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05888c3b-d5fd-44d3-9af8-32c6533205f4 admin admin] PATH_INFO: `/v3/users/9775b7c413ef4ab7b5c46a62b772df4d` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:21.801669 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05888c3b-d5fd-44d3-9af8-32c6533205f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9775b7c413ef4ab7b5c46a62b772df4d)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:21.802766 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05888c3b-d5fd-44d3-9af8-32c6533205f4 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:21.821664 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:21.903914 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-36fb12ae-f33a-40b8-8ce2-ba0948fd7235 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdd38fafaf2148759e10988f3293b792', 'request_id': 'req-36fb12ae-f33a-40b8-8ce2-ba0948fd7235'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:21.909791 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-36fb12ae-f33a-40b8-8ce2-ba0948fd7235 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdd38fafaf2148759e10988f3293b792', 'request_id': 'req-36fb12ae-f33a-40b8-8ce2-ba0948fd7235'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:21.995863 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-36fb12ae-f33a-40b8-8ce2-ba0948fd7235 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdd38fafaf2148759e10988f3293b792', 'request_id': 'req-36fb12ae-f33a-40b8-8ce2-ba0948fd7235'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:22.009963 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-36fb12ae-f33a-40b8-8ce2-ba0948fd7235 admin admin] The token cache is being invalidate because project fdd38fafaf2148759e10988f3293b792 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81746) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:21:22.018507 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-36fb12ae-f33a-40b8-8ce2-ba0948fd7235 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:22.018507 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-36fb12ae-f33a-40b8-8ce2-ba0948fd7235 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:22.018507 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 936/1878] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:21:21 2022] DELETE /identity/v3/projects/fdd38fafaf2148759e10988f3293b792 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:21:22.019114 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-05888c3b-d5fd-44d3-9af8-32c6533205f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9775b7c413ef4ab7b5c46a62b772df4d'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:22.019529 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fb12ae-f33a-40b8-8ce2-ba0948fd7235 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:22.019529 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-36fb12ae-f33a-40b8-8ce2-ba0948fd7235 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:22.019981 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-05888c3b-d5fd-44d3-9af8-32c6533205f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9775b7c413ef4ab7b5c46a62b772df4d'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:22.034714 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-05888c3b-d5fd-44d3-9af8-32c6533205f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9775b7c413ef4ab7b5c46a62b772df4d'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:22.080700 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fb12ae-f33a-40b8-8ce2-ba0948fd7235 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:22.085553 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c574a35b-9c4c-4764-b4c8-96eb48e8b14f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:22.086160 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c574a35b-9c4c-4764-b4c8-96eb48e8b14f admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:22.086442 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c574a35b-9c4c-4764-b4c8-96eb48e8b14f admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:22.087039 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c574a35b-9c4c-4764-b4c8-96eb48e8b14f admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:22.087395 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c574a35b-9c4c-4764-b4c8-96eb48e8b14f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:22.089025 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c574a35b-9c4c-4764-b4c8-96eb48e8b14f admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:22.099071 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 937/1879] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:21:22 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:21:22.099923 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c574a35b-9c4c-4764-b4c8-96eb48e8b14f admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:22.101585 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-c574a35b-9c4c-4764-b4c8-96eb48e8b14f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:22.114021 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c574a35b-9c4c-4764-b4c8-96eb48e8b14f admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:22.119237 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233ef3e7-ca69-4dc2-a3de-36a893775a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:22.120397 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233ef3e7-ca69-4dc2-a3de-36a893775a95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:22.120678 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233ef3e7-ca69-4dc2-a3de-36a893775a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:22.120942 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233ef3e7-ca69-4dc2-a3de-36a893775a95 admin admin] PATH_INFO: `/v3/users/d9be653da6c1438dbb8dc202339a16af` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:22.122074 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233ef3e7-ca69-4dc2-a3de-36a893775a95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9be653da6c1438dbb8dc202339a16af)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:22.124125 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233ef3e7-ca69-4dc2-a3de-36a893775a95 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:22.149036 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-05888c3b-d5fd-44d3-9af8-32c6533205f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9775b7c413ef4ab7b5c46a62b772df4d'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:22.149674 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-05888c3b-d5fd-44d3-9af8-32c6533205f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9775b7c413ef4ab7b5c46a62b772df4d'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:22.162823 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 943/1880] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:21:21 2022] DELETE /identity/v3/users/9775b7c413ef4ab7b5c46a62b772df4d => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:21:22.166116 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05888c3b-d5fd-44d3-9af8-32c6533205f4 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:22.166116 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-05888c3b-d5fd-44d3-9af8-32c6533205f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:22.232758 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05888c3b-d5fd-44d3-9af8-32c6533205f4 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:22.241225 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ecdddd-54f7-4045-bc8b-3dfa625ff415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:22.241847 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ecdddd-54f7-4045-bc8b-3dfa625ff415 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:22.242208 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ecdddd-54f7-4045-bc8b-3dfa625ff415 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:22.242491 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ecdddd-54f7-4045-bc8b-3dfa625ff415 admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:22.243210 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ecdddd-54f7-4045-bc8b-3dfa625ff415 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:22.245520 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ecdddd-54f7-4045-bc8b-3dfa625ff415 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:22.256029 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 944/1881] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:21:22 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:21:22.260070 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ecdddd-54f7-4045-bc8b-3dfa625ff415 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:22.270877 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ecdddd-54f7-4045-bc8b-3dfa625ff415 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:22.275842 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525b9e91-161e-44d5-a31c-4b0a84e76b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:22.276505 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b9e91-161e-44d5-a31c-4b0a84e76b72 admin admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:22.276780 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b9e91-161e-44d5-a31c-4b0a84e76b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:22.277048 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b9e91-161e-44d5-a31c-4b0a84e76b72 admin admin] PATH_INFO: `/v3/projects/6dcbbe658be341c18d51a0ce90de0827/users/5f35aec2249f4a36b0d3f7408a6aaf25/roles/ef689f96442945e8a7bc8cbfee13d909` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:22.292798 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525b9e91-161e-44d5-a31c-4b0a84e76b72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6dcbbe658be341c18d51a0ce90de0827, user_id=5f35aec2249f4a36b0d3f7408a6aaf25, role_id=ef689f96442945e8a7bc8cbfee13d909)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:22.295658 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525b9e91-161e-44d5-a31c-4b0a84e76b72 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:22.368008 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-233ef3e7-ca69-4dc2-a3de-36a893775a95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9be653da6c1438dbb8dc202339a16af'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:22.368967 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-233ef3e7-ca69-4dc2-a3de-36a893775a95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9be653da6c1438dbb8dc202339a16af'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:22.382500 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-233ef3e7-ca69-4dc2-a3de-36a893775a95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9be653da6c1438dbb8dc202339a16af'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:22.386877 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:22.395520 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 945/1882] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:21:22 2022] PUT /identity/v3/projects/6dcbbe658be341c18d51a0ce90de0827/users/5f35aec2249f4a36b0d3f7408a6aaf25/roles/ef689f96442945e8a7bc8cbfee13d909 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 00:21:22.405778 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7c4ffd-5960-406b-998c-428371cdd28f None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:22.406149 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7c4ffd-5960-406b-998c-428371cdd28f None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:22.406977 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7c4ffd-5960-406b-998c-428371cdd28f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:22.437777 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-7a7c4ffd-5960-406b-998c-428371cdd28f None None] MFA Rules not processed for user `5f35aec2249f4a36b0d3f7408a6aaf25`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:21:22.439008 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-7a7c4ffd-5960-406b-998c-428371cdd28f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:22.484499 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-233ef3e7-ca69-4dc2-a3de-36a893775a95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9be653da6c1438dbb8dc202339a16af'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:22.485539 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-233ef3e7-ca69-4dc2-a3de-36a893775a95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9be653da6c1438dbb8dc202339a16af'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:22.500972 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 938/1883] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:21:22 2022] DELETE /identity/v3/users/d9be653da6c1438dbb8dc202339a16af => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:21:22.517734 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 946/1884] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:21:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 00:21:22.529607 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233ef3e7-ca69-4dc2-a3de-36a893775a95 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:22.531065 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-233ef3e7-ca69-4dc2-a3de-36a893775a95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:22.603429 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233ef3e7-ca69-4dc2-a3de-36a893775a95 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:22.608178 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a675cb-274a-4e84-b6e1-9a688f5303ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:22.608796 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a675cb-274a-4e84-b6e1-9a688f5303ab service cinder] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:22.609070 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a675cb-274a-4e84-b6e1-9a688f5303ab service cinder] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:22.609339 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a675cb-274a-4e84-b6e1-9a688f5303ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:22.677362 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a675cb-274a-4e84-b6e1-9a688f5303ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:22.684668 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a675cb-274a-4e84-b6e1-9a688f5303ab service cinder] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:22.686284 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7c4ffd-5960-406b-998c-428371cdd28f None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:22.687864 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-7a7c4ffd-5960-406b-998c-428371cdd28f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:22.707454 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 939/1885] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:21:22 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:21:22.730661 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a675cb-274a-4e84-b6e1-9a688f5303ab service cinder] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:22.735028 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-d3a675cb-274a-4e84-b6e1-9a688f5303ab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:22.806682 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7c4ffd-5960-406b-998c-428371cdd28f None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:22.865093 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a675cb-274a-4e84-b6e1-9a688f5303ab service cinder] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:22.871062 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:22.873850 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d7bf1e-8612-4902-8fdf-52085d2d154d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:22.875586 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d7bf1e-8612-4902-8fdf-52085d2d154d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:22.875945 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d7bf1e-8612-4902-8fdf-52085d2d154d admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:22.877036 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d7bf1e-8612-4902-8fdf-52085d2d154d admin admin] PATH_INFO: `/v3/users/5f35aec2249f4a36b0d3f7408a6aaf25` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:22.878899 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d7bf1e-8612-4902-8fdf-52085d2d154d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f35aec2249f4a36b0d3f7408a6aaf25)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:22.882549 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d7bf1e-8612-4902-8fdf-52085d2d154d admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:22.910342 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65924f58-08d0-4f9a-bb8d-f7d383138685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:22.911294 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65924f58-08d0-4f9a-bb8d-f7d383138685 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:22.911735 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65924f58-08d0-4f9a-bb8d-f7d383138685 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:22.912154 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65924f58-08d0-4f9a-bb8d-f7d383138685 admin admin] PATH_INFO: `/v3/projects/82538d41a63b49a282cba8ddaa19cdc0` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:22.916896 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65924f58-08d0-4f9a-bb8d-f7d383138685 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82538d41a63b49a282cba8ddaa19cdc0)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:22.919142 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65924f58-08d0-4f9a-bb8d-f7d383138685 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:23.111749 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-64d7bf1e-8612-4902-8fdf-52085d2d154d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f35aec2249f4a36b0d3f7408a6aaf25'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:23.112176 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-65924f58-08d0-4f9a-bb8d-f7d383138685 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82538d41a63b49a282cba8ddaa19cdc0', 'request_id': 'req-65924f58-08d0-4f9a-bb8d-f7d383138685'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:23.112529 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-64d7bf1e-8612-4902-8fdf-52085d2d154d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f35aec2249f4a36b0d3f7408a6aaf25'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:23.120760 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-65924f58-08d0-4f9a-bb8d-f7d383138685 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82538d41a63b49a282cba8ddaa19cdc0', 'request_id': 'req-65924f58-08d0-4f9a-bb8d-f7d383138685'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:23.130097 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-64d7bf1e-8612-4902-8fdf-52085d2d154d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f35aec2249f4a36b0d3f7408a6aaf25'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:23.218170 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-65924f58-08d0-4f9a-bb8d-f7d383138685 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82538d41a63b49a282cba8ddaa19cdc0', 'request_id': 'req-65924f58-08d0-4f9a-bb8d-f7d383138685'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:23.231406 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-65924f58-08d0-4f9a-bb8d-f7d383138685 admin admin] The token cache is being invalidate because project 82538d41a63b49a282cba8ddaa19cdc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81747) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:21:23.231658 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-65924f58-08d0-4f9a-bb8d-f7d383138685 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:23.233651 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-65924f58-08d0-4f9a-bb8d-f7d383138685 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:23.236185 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 947/1886] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:21:22 2022] DELETE /identity/v3/projects/82538d41a63b49a282cba8ddaa19cdc0 => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 00:21:23.237152 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65924f58-08d0-4f9a-bb8d-f7d383138685 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:23.241398 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-65924f58-08d0-4f9a-bb8d-f7d383138685 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:23.297832 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-64d7bf1e-8612-4902-8fdf-52085d2d154d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f35aec2249f4a36b0d3f7408a6aaf25'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:23.298316 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-64d7bf1e-8612-4902-8fdf-52085d2d154d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f35aec2249f4a36b0d3f7408a6aaf25'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:23.315829 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 940/1887] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:21:22 2022] DELETE /identity/v3/users/5f35aec2249f4a36b0d3f7408a6aaf25 => generated 0 bytes in 586 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:21:23.321682 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3840b5-6c21-4c4a-bf7a-eea74b69deab None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:23.322048 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3840b5-6c21-4c4a-bf7a-eea74b69deab None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:23.322227 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3840b5-6c21-4c4a-bf7a-eea74b69deab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:23.344028 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-1d3840b5-6c21-4c4a-bf7a-eea74b69deab None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:21:23.345192 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-1d3840b5-6c21-4c4a-bf7a-eea74b69deab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:23.348282 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65924f58-08d0-4f9a-bb8d-f7d383138685 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:23.415555 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849b01b4-f787-4e68-8066-8bdaff9e146a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:23.416811 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849b01b4-f787-4e68-8066-8bdaff9e146a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:23.417266 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849b01b4-f787-4e68-8066-8bdaff9e146a admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:23.417549 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849b01b4-f787-4e68-8066-8bdaff9e146a admin admin] PATH_INFO: `/v3/projects/84532b956aaf4d3a9d3a022d5c1e7e66` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:23.418698 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849b01b4-f787-4e68-8066-8bdaff9e146a admin admin] RBAC: Authorizing `identity:delete_project(project_id=84532b956aaf4d3a9d3a022d5c1e7e66)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:23.420670 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849b01b4-f787-4e68-8066-8bdaff9e146a admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:23.423107 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 941/1888] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:21:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:21:23.425014 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d998caa-1638-4400-938a-55b850a0820d None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:23.425513 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d998caa-1638-4400-938a-55b850a0820d None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:23.425964 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d998caa-1638-4400-938a-55b850a0820d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:23.468865 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-9d998caa-1638-4400-938a-55b850a0820d None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:21:23.471068 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-9d998caa-1638-4400-938a-55b850a0820d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:23.485995 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 942/1889] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:21:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:21:23.485995 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfd13f0-9b28-4680-939d-4eabd12d16cb None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:23.485995 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfd13f0-9b28-4680-939d-4eabd12d16cb None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:23.485995 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfd13f0-9b28-4680-939d-4eabd12d16cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:23.511041 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-9dfd13f0-9b28-4680-939d-4eabd12d16cb None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:21:23.512355 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-9dfd13f0-9b28-4680-939d-4eabd12d16cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:23.520915 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 943/1890] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:21:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:21:23.528109 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfd13f0-9b28-4680-939d-4eabd12d16cb None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:23.528109 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-9dfd13f0-9b28-4680-939d-4eabd12d16cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:23.615267 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfd13f0-9b28-4680-939d-4eabd12d16cb None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:23.619798 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-849b01b4-f787-4e68-8066-8bdaff9e146a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84532b956aaf4d3a9d3a022d5c1e7e66', 'request_id': 'req-849b01b4-f787-4e68-8066-8bdaff9e146a'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:23.640298 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-849b01b4-f787-4e68-8066-8bdaff9e146a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84532b956aaf4d3a9d3a022d5c1e7e66', 'request_id': 'req-849b01b4-f787-4e68-8066-8bdaff9e146a'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:23.706545 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1152528b-268c-4559-bf32-fa3ce3bffff2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:23.707175 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1152528b-268c-4559-bf32-fa3ce3bffff2 service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:23.707479 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1152528b-268c-4559-bf32-fa3ce3bffff2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:23.707965 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1152528b-268c-4559-bf32-fa3ce3bffff2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:23.723501 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-849b01b4-f787-4e68-8066-8bdaff9e146a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84532b956aaf4d3a9d3a022d5c1e7e66', 'request_id': 'req-849b01b4-f787-4e68-8066-8bdaff9e146a'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:23.740570 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-849b01b4-f787-4e68-8066-8bdaff9e146a admin admin] The token cache is being invalidate because project 84532b956aaf4d3a9d3a022d5c1e7e66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81747) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:21:23.745665 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-849b01b4-f787-4e68-8066-8bdaff9e146a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:23.746447 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-849b01b4-f787-4e68-8066-8bdaff9e146a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:23.748343 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 948/1891] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:21:23 2022] DELETE /identity/v3/projects/84532b956aaf4d3a9d3a022d5c1e7e66 => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:21:23.748853 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849b01b4-f787-4e68-8066-8bdaff9e146a admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:23.750494 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-849b01b4-f787-4e68-8066-8bdaff9e146a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:23.790112 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1152528b-268c-4559-bf32-fa3ce3bffff2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:23.792211 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1152528b-268c-4559-bf32-fa3ce3bffff2 service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:23.820520 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 944/1892] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:21:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:21:23.825672 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8bd30d-809b-406b-a0e2-635e99a44ae6 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:23.825968 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8bd30d-809b-406b-a0e2-635e99a44ae6 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:23.826239 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8bd30d-809b-406b-a0e2-635e99a44ae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:23.870065 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849b01b4-f787-4e68-8066-8bdaff9e146a admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:23.871119 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-2f8bd30d-809b-406b-a0e2-635e99a44ae6 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:21:23.872839 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-2f8bd30d-809b-406b-a0e2-635e99a44ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:23.876919 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94220722-b205-4e6d-bc9d-1a39443eda89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:23.877648 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94220722-b205-4e6d-bc9d-1a39443eda89 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:23.877932 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94220722-b205-4e6d-bc9d-1a39443eda89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:23.878214 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94220722-b205-4e6d-bc9d-1a39443eda89 admin admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:23.879716 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94220722-b205-4e6d-bc9d-1a39443eda89 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:21:23.879716 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94220722-b205-4e6d-bc9d-1a39443eda89 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:23.887435 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94220722-b205-4e6d-bc9d-1a39443eda89 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:23.915487 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 945/1893] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:21:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:21:23.923704 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd7ed3d-ec8c-4307-ae79-5a570e74121e None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:23.924195 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd7ed3d-ec8c-4307-ae79-5a570e74121e None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:23.925148 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd7ed3d-ec8c-4307-ae79-5a570e74121e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:23.927187 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 949/1894] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Oct 12 00:21:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:21:23.928205 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94220722-b205-4e6d-bc9d-1a39443eda89 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:23.946058 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94220722-b205-4e6d-bc9d-1a39443eda89 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:23.960263 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3db0e6-1bac-4779-9e3b-b2daa093acb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:23.961603 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3db0e6-1bac-4779-9e3b-b2daa093acb0 admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:23.961603 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3db0e6-1bac-4779-9e3b-b2daa093acb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:23.961823 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3db0e6-1bac-4779-9e3b-b2daa093acb0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:23.966908 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3db0e6-1bac-4779-9e3b-b2daa093acb0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:23.967884 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-dfd7ed3d-ec8c-4307-ae79-5a570e74121e None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:21:23.970310 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3db0e6-1bac-4779-9e3b-b2daa093acb0 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:23.974151 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-dfd7ed3d-ec8c-4307-ae79-5a570e74121e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:23.991109 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 946/1895] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:21:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:21:23.994995 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd7ed3d-ec8c-4307-ae79-5a570e74121e None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:24.006766 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd7ed3d-ec8c-4307-ae79-5a570e74121e None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:24.013811 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241c3ed7-e8bb-4e0c-9f04-01b3409fa073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:24.014157 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241c3ed7-e8bb-4e0c-9f04-01b3409fa073 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:24.014920 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241c3ed7-e8bb-4e0c-9f04-01b3409fa073 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:24.015121 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241c3ed7-e8bb-4e0c-9f04-01b3409fa073 admin admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:24.015907 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241c3ed7-e8bb-4e0c-9f04-01b3409fa073 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:21:24.016472 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241c3ed7-e8bb-4e0c-9f04-01b3409fa073 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:24.018174 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241c3ed7-e8bb-4e0c-9f04-01b3409fa073 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:24.040320 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 947/1896] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Oct 12 00:21:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:21:24.042821 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241c3ed7-e8bb-4e0c-9f04-01b3409fa073 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:24.056305 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241c3ed7-e8bb-4e0c-9f04-01b3409fa073 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:24.061179 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9970db72-9e91-4f3f-aa4c-916603dc7cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:24.062009 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9970db72-9e91-4f3f-aa4c-916603dc7cec admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:24.068465 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9970db72-9e91-4f3f-aa4c-916603dc7cec admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:24.068465 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9970db72-9e91-4f3f-aa4c-916603dc7cec admin admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:24.068465 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9970db72-9e91-4f3f-aa4c-916603dc7cec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:24.068465 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9970db72-9e91-4f3f-aa4c-916603dc7cec admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:24.123233 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 950/1897] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:21:23 2022] POST /identity/v3/projects => generated 385 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:21:24.124157 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3db0e6-1bac-4779-9e3b-b2daa093acb0 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:24.177895 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 948/1898] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:21:24 2022] POST /identity/v3/projects => generated 371 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 00:21:24.182895 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9970db72-9e91-4f3f-aa4c-916603dc7cec admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:24.249921 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3db0e6-1bac-4779-9e3b-b2daa093acb0 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:24.284633 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:24.288064 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9970db72-9e91-4f3f-aa4c-916603dc7cec admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:24.293677 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d611180a-a1dd-4f67-bdae-3a511d712dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:24.294429 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d611180a-a1dd-4f67-bdae-3a511d712dd0 admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:24.294828 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d611180a-a1dd-4f67-bdae-3a511d712dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:24.295245 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d611180a-a1dd-4f67-bdae-3a511d712dd0 admin admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:24.296366 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d611180a-a1dd-4f67-bdae-3a511d712dd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:24.299299 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d611180a-a1dd-4f67-bdae-3a511d712dd0 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:24.328038 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd257d53-671a-46bc-834d-229171e79e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:24.328829 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd257d53-671a-46bc-834d-229171e79e3d admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:24.329183 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd257d53-671a-46bc-834d-229171e79e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:24.329526 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd257d53-671a-46bc-834d-229171e79e3d admin admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:24.330238 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd257d53-671a-46bc-834d-229171e79e3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:24.332732 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd257d53-671a-46bc-834d-229171e79e3d admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:24.427458 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 949/1899] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:21:24 2022] POST /identity/v3/users => generated 286 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 00:21:24.427458 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d611180a-a1dd-4f67-bdae-3a511d712dd0 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:24.427458 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-d611180a-a1dd-4f67-bdae-3a511d712dd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:24.441837 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d611180a-a1dd-4f67-bdae-3a511d712dd0 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:24.454226 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 951/1900] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:21:24 2022] POST /identity/v3/users => generated 293 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:21:24.454226 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd257d53-671a-46bc-834d-229171e79e3d admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:24.454497 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc45e1b-68af-48c5-9c1e-069f6abdca88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:24.455416 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc45e1b-68af-48c5-9c1e-069f6abdca88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:24.455731 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc45e1b-68af-48c5-9c1e-069f6abdca88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:24.456028 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc45e1b-68af-48c5-9c1e-069f6abdca88 admin admin] PATH_INFO: `/v3/projects/6dcbbe658be341c18d51a0ce90de0827` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:24.457222 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc45e1b-68af-48c5-9c1e-069f6abdca88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6dcbbe658be341c18d51a0ce90de0827)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:24.459152 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc45e1b-68af-48c5-9c1e-069f6abdca88 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:24.464598 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd257d53-671a-46bc-834d-229171e79e3d admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:24.469866 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ab688c-c311-469c-88f7-efb43b729e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:24.470595 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab688c-c311-469c-88f7-efb43b729e12 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:24.470866 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab688c-c311-469c-88f7-efb43b729e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:24.471134 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab688c-c311-469c-88f7-efb43b729e12 admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:24.471813 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ab688c-c311-469c-88f7-efb43b729e12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:24.473575 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ab688c-c311-469c-88f7-efb43b729e12 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:24.484464 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 952/1901] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:21:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:21:24.485317 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ab688c-c311-469c-88f7-efb43b729e12 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:24.497694 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ab688c-c311-469c-88f7-efb43b729e12 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:24.507633 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:24.507633 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3a1eab-d3b7-4db0-837f-772bad979acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:24.507633 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3a1eab-d3b7-4db0-837f-772bad979acc admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:24.507633 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3a1eab-d3b7-4db0-837f-772bad979acc admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:24.508012 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3a1eab-d3b7-4db0-837f-772bad979acc admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:24.508012 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3a1eab-d3b7-4db0-837f-772bad979acc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:24.509042 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3a1eab-d3b7-4db0-837f-772bad979acc admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:24.521048 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 953/1902] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:21:24 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:21:24.521898 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3a1eab-d3b7-4db0-837f-772bad979acc admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:24.533790 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3a1eab-d3b7-4db0-837f-772bad979acc admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:24.539537 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e03fda5-bf10-4ef5-99cc-f3d16afba82b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:24.540168 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e03fda5-bf10-4ef5-99cc-f3d16afba82b admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:24.540499 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e03fda5-bf10-4ef5-99cc-f3d16afba82b admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:24.540849 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e03fda5-bf10-4ef5-99cc-f3d16afba82b admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:24.542202 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e03fda5-bf10-4ef5-99cc-f3d16afba82b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:24.543882 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e03fda5-bf10-4ef5-99cc-f3d16afba82b admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:24.560240 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 954/1903] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:21:24 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:21:24.561233 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e03fda5-bf10-4ef5-99cc-f3d16afba82b admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:24.579052 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e03fda5-bf10-4ef5-99cc-f3d16afba82b admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:24.579254 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-3fc45e1b-68af-48c5-9c1e-069f6abdca88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6dcbbe658be341c18d51a0ce90de0827', 'request_id': 'req-3fc45e1b-68af-48c5-9c1e-069f6abdca88'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:24.586787 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-3fc45e1b-68af-48c5-9c1e-069f6abdca88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6dcbbe658be341c18d51a0ce90de0827', 'request_id': 'req-3fc45e1b-68af-48c5-9c1e-069f6abdca88'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:24.652565 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-3fc45e1b-68af-48c5-9c1e-069f6abdca88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6dcbbe658be341c18d51a0ce90de0827', 'request_id': 'req-3fc45e1b-68af-48c5-9c1e-069f6abdca88'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:24.668976 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-3fc45e1b-68af-48c5-9c1e-069f6abdca88 admin admin] The token cache is being invalidate because project 6dcbbe658be341c18d51a0ce90de0827 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81746) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:21:24.669271 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-3fc45e1b-68af-48c5-9c1e-069f6abdca88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:24.670603 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f16036a-a389-4293-9a37-52780b3e63ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:24.671240 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f16036a-a389-4293-9a37-52780b3e63ba admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:24.671544 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f16036a-a389-4293-9a37-52780b3e63ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:24.671810 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f16036a-a389-4293-9a37-52780b3e63ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:24.672339 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-3fc45e1b-68af-48c5-9c1e-069f6abdca88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:24.673160 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f16036a-a389-4293-9a37-52780b3e63ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:24.675743 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f16036a-a389-4293-9a37-52780b3e63ba admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:24.676221 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 950/1904] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:21:24 2022] DELETE /identity/v3/projects/6dcbbe658be341c18d51a0ce90de0827 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:21:24.676727 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc45e1b-68af-48c5-9c1e-069f6abdca88 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:24.682538 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-3fc45e1b-68af-48c5-9c1e-069f6abdca88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:24.696602 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 955/1905] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:21:24 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:21:24.699744 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179cdfa6-d20d-461f-97f8-2cb2c17f36e9 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:24.700525 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179cdfa6-d20d-461f-97f8-2cb2c17f36e9 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:24.701668 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179cdfa6-d20d-461f-97f8-2cb2c17f36e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:24.703416 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc45e1b-68af-48c5-9c1e-069f6abdca88 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:24.710082 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53b3f99-06ce-4b71-9ff0-689cc1edf87c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:24.710772 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b3f99-06ce-4b71-9ff0-689cc1edf87c admin admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:24.711048 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b3f99-06ce-4b71-9ff0-689cc1edf87c admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:24.711315 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b3f99-06ce-4b71-9ff0-689cc1edf87c admin admin] PATH_INFO: `/v3/projects/ac3fc5ff2b464891ba5fd114cea03198/users/de2dff9816a943afb842a41d467d8019/roles/ef689f96442945e8a7bc8cbfee13d909` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:24.747509 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-179cdfa6-d20d-461f-97f8-2cb2c17f36e9 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:21:24.749447 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-179cdfa6-d20d-461f-97f8-2cb2c17f36e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:24.760543 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 956/1906] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:21:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:21:24.761690 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179cdfa6-d20d-461f-97f8-2cb2c17f36e9 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:24.766508 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-179cdfa6-d20d-461f-97f8-2cb2c17f36e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:24.772315 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53b3f99-06ce-4b71-9ff0-689cc1edf87c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac3fc5ff2b464891ba5fd114cea03198, user_id=de2dff9816a943afb842a41d467d8019, role_id=ef689f96442945e8a7bc8cbfee13d909)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:24.775207 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53b3f99-06ce-4b71-9ff0-689cc1edf87c admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:24.782929 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179cdfa6-d20d-461f-97f8-2cb2c17f36e9 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:24.789715 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48076a0b-6865-4fe9-9c4d-49bd76029ce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:24.790467 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48076a0b-6865-4fe9-9c4d-49bd76029ce4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:24.790902 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48076a0b-6865-4fe9-9c4d-49bd76029ce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:24.791911 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48076a0b-6865-4fe9-9c4d-49bd76029ce4 admin admin] PATH_INFO: `/v3/projects/5a073bf9f0974dcdba242b6fce21e71e/users/f5a57ad052d746b28c87030a10f3f8ed/roles/ef689f96442945e8a7bc8cbfee13d909` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:24.812243 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48076a0b-6865-4fe9-9c4d-49bd76029ce4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a073bf9f0974dcdba242b6fce21e71e, user_id=f5a57ad052d746b28c87030a10f3f8ed, role_id=ef689f96442945e8a7bc8cbfee13d909)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:24.815036 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48076a0b-6865-4fe9-9c4d-49bd76029ce4 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:24.890409 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 951/1907] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:21:24 2022] PUT /identity/v3/projects/ac3fc5ff2b464891ba5fd114cea03198/users/de2dff9816a943afb842a41d467d8019/roles/ef689f96442945e8a7bc8cbfee13d909 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:21:24.890409 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4e2733-2b52-4a37-ac2f-55429fbd17f2 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:24.890409 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4e2733-2b52-4a37-ac2f-55429fbd17f2 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:24.890409 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4e2733-2b52-4a37-ac2f-55429fbd17f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:24.902261 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 957/1908] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:21:24 2022] PUT /identity/v3/projects/5a073bf9f0974dcdba242b6fce21e71e/users/f5a57ad052d746b28c87030a10f3f8ed/roles/ef689f96442945e8a7bc8cbfee13d909 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:21:24.903586 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab0d1d3-1e26-4efb-b4b8-15b92ce6e072 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:24.908214 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab0d1d3-1e26-4efb-b4b8-15b92ce6e072 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:24.908214 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab0d1d3-1e26-4efb-b4b8-15b92ce6e072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:24.923966 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-2e4e2733-2b52-4a37-ac2f-55429fbd17f2 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:21:24.924952 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-2e4e2733-2b52-4a37-ac2f-55429fbd17f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:24.973635 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-fab0d1d3-1e26-4efb-b4b8-15b92ce6e072 None None] MFA Rules not processed for user `de2dff9816a943afb842a41d467d8019`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:21:24.978524 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-fab0d1d3-1e26-4efb-b4b8-15b92ce6e072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:25.050078 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 952/1909] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:21:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:21:25.051423 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b99f852-6944-4e47-bab2-5bd95cb5e827 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:25.051787 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b99f852-6944-4e47-bab2-5bd95cb5e827 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:25.052060 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b99f852-6944-4e47-bab2-5bd95cb5e827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:25.085282 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-1b99f852-6944-4e47-bab2-5bd95cb5e827 None None] MFA Rules not processed for user `f5a57ad052d746b28c87030a10f3f8ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:21:25.086495 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-1b99f852-6944-4e47-bab2-5bd95cb5e827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:25.100266 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 958/1910] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:21:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:21:25.101269 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab0d1d3-1e26-4efb-b4b8-15b92ce6e072 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:25.123387 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab0d1d3-1e26-4efb-b4b8-15b92ce6e072 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:25.134773 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005136cb-3f72-4d53-a46a-955a0f7553b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:25.135460 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005136cb-3f72-4d53-a46a-955a0f7553b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:25.135738 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005136cb-3f72-4d53-a46a-955a0f7553b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:25.136005 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005136cb-3f72-4d53-a46a-955a0f7553b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:25.136657 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005136cb-3f72-4d53-a46a-955a0f7553b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:21:25.136944 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005136cb-3f72-4d53-a46a-955a0f7553b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:25.138811 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005136cb-3f72-4d53-a46a-955a0f7553b8 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:25.155812 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 959/1911] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Oct 12 00:21:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:21:25.156770 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005136cb-3f72-4d53-a46a-955a0f7553b8 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:25.158651 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-005136cb-3f72-4d53-a46a-955a0f7553b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:25.193947 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 953/1912] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:21:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:21:25.195060 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b99f852-6944-4e47-bab2-5bd95cb5e827 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:25.207292 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b99f852-6944-4e47-bab2-5bd95cb5e827 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:25.212281 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edcdd64-c9f5-421d-9109-e3eba9796e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:25.213764 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edcdd64-c9f5-421d-9109-e3eba9796e5d admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:25.214282 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edcdd64-c9f5-421d-9109-e3eba9796e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:25.214654 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edcdd64-c9f5-421d-9109-e3eba9796e5d admin admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:25.215696 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edcdd64-c9f5-421d-9109-e3eba9796e5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:25.217665 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edcdd64-c9f5-421d-9109-e3eba9796e5d admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:25.245112 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005136cb-3f72-4d53-a46a-955a0f7553b8 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:25.249059 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869aff6b-53cf-4df7-b9f9-087f2000a237 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:25.249798 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869aff6b-53cf-4df7-b9f9-087f2000a237 service cinder] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:25.250132 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869aff6b-53cf-4df7-b9f9-087f2000a237 service cinder] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:25.253374 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869aff6b-53cf-4df7-b9f9-087f2000a237 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:25.260497 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869aff6b-53cf-4df7-b9f9-087f2000a237 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:25.262612 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869aff6b-53cf-4df7-b9f9-087f2000a237 service cinder] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:25.307458 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 960/1913] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:21:25 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:21:25.308534 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869aff6b-53cf-4df7-b9f9-087f2000a237 service cinder] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:25.310431 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-869aff6b-53cf-4df7-b9f9-087f2000a237 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:25.315350 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 954/1914] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:21:25 2022] POST /identity/v3/projects => generated 383 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:21:25.316713 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edcdd64-c9f5-421d-9109-e3eba9796e5d admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:25.329176 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869aff6b-53cf-4df7-b9f9-087f2000a237 service cinder] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:25.416316 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56348dd0-4ad9-47c9-b5f6-d8035321b4ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:25.417018 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56348dd0-4ad9-47c9-b5f6-d8035321b4ed service cinder] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:25.417368 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56348dd0-4ad9-47c9-b5f6-d8035321b4ed service cinder] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:25.417884 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56348dd0-4ad9-47c9-b5f6-d8035321b4ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:25.454300 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edcdd64-c9f5-421d-9109-e3eba9796e5d admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:25.474252 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aade7f99-c92d-4ee2-bc9d-f340e4fdd2d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:25.474902 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade7f99-c92d-4ee2-bc9d-f340e4fdd2d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:25.475178 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade7f99-c92d-4ee2-bc9d-f340e4fdd2d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:25.475494 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade7f99-c92d-4ee2-bc9d-f340e4fdd2d2 admin admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:25.476147 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aade7f99-c92d-4ee2-bc9d-f340e4fdd2d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:25.478384 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aade7f99-c92d-4ee2-bc9d-f340e4fdd2d2 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:25.546593 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:25.550844 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56348dd0-4ad9-47c9-b5f6-d8035321b4ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:25.552333 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56348dd0-4ad9-47c9-b5f6-d8035321b4ed service cinder] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:25.565479 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 961/1915] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:21:25 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:21:25.566450 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56348dd0-4ad9-47c9-b5f6-d8035321b4ed service cinder] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:25.580058 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56348dd0-4ad9-47c9-b5f6-d8035321b4ed service cinder] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:25.585767 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4439fca-57dc-4137-91ab-1080d32a1445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:25.586595 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4439fca-57dc-4137-91ab-1080d32a1445 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:25.586876 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4439fca-57dc-4137-91ab-1080d32a1445 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:25.591912 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4439fca-57dc-4137-91ab-1080d32a1445 admin admin] PATH_INFO: `/v3/users/de2dff9816a943afb842a41d467d8019` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:25.591912 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4439fca-57dc-4137-91ab-1080d32a1445 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de2dff9816a943afb842a41d467d8019)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:25.591912 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4439fca-57dc-4137-91ab-1080d32a1445 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:25.597782 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 955/1916] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:21:25 2022] POST /identity/v3/users => generated 292 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:21:25.603216 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aade7f99-c92d-4ee2-bc9d-f340e4fdd2d2 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:25.637248 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aade7f99-c92d-4ee2-bc9d-f340e4fdd2d2 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:25.648051 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3430af3f-9f4c-4330-a29c-1e9f3eb21af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:25.648729 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3430af3f-9f4c-4330-a29c-1e9f3eb21af5 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:25.649133 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3430af3f-9f4c-4330-a29c-1e9f3eb21af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:25.649473 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3430af3f-9f4c-4330-a29c-1e9f3eb21af5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:25.650242 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3430af3f-9f4c-4330-a29c-1e9f3eb21af5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:25.652058 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3430af3f-9f4c-4330-a29c-1e9f3eb21af5 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:25.680032 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 956/1917] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:21:25 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:21:25.684362 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3430af3f-9f4c-4330-a29c-1e9f3eb21af5 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:25.698742 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3430af3f-9f4c-4330-a29c-1e9f3eb21af5 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:25.705059 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c6e819-dc6c-4216-b927-22f1b4917d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:25.705866 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c6e819-dc6c-4216-b927-22f1b4917d0a admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:25.706219 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c6e819-dc6c-4216-b927-22f1b4917d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:25.706708 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c6e819-dc6c-4216-b927-22f1b4917d0a admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:25.707459 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c6e819-dc6c-4216-b927-22f1b4917d0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:25.709163 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c6e819-dc6c-4216-b927-22f1b4917d0a admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:25.729034 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 957/1918] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:21:25 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 00:21:25.734165 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c6e819-dc6c-4216-b927-22f1b4917d0a admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:25.751963 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c6e819-dc6c-4216-b927-22f1b4917d0a admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:25.757750 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145c7494-1682-4234-9665-b98020d4c3a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:25.758623 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c7494-1682-4234-9665-b98020d4c3a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:25.759080 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c7494-1682-4234-9665-b98020d4c3a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:25.759644 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c7494-1682-4234-9665-b98020d4c3a5 admin admin] PATH_INFO: `/v3/projects/4194baa4b1fe4ab2840720dbf012a2f3/users/b0791b4507cf48bca517f41f0feebd78/roles/ef689f96442945e8a7bc8cbfee13d909` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:25.798450 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145c7494-1682-4234-9665-b98020d4c3a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4194baa4b1fe4ab2840720dbf012a2f3, user_id=b0791b4507cf48bca517f41f0feebd78, role_id=ef689f96442945e8a7bc8cbfee13d909)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:25.802801 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145c7494-1682-4234-9665-b98020d4c3a5 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:25.814807 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-b4439fca-57dc-4137-91ab-1080d32a1445 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de2dff9816a943afb842a41d467d8019'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:25.817280 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-b4439fca-57dc-4137-91ab-1080d32a1445 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de2dff9816a943afb842a41d467d8019'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:25.838319 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-b4439fca-57dc-4137-91ab-1080d32a1445 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de2dff9816a943afb842a41d467d8019'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:25.929872 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-b4439fca-57dc-4137-91ab-1080d32a1445 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de2dff9816a943afb842a41d467d8019'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:25.933840 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-b4439fca-57dc-4137-91ab-1080d32a1445 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de2dff9816a943afb842a41d467d8019'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:25.935792 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 958/1919] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:21:25 2022] PUT /identity/v3/projects/4194baa4b1fe4ab2840720dbf012a2f3/users/b0791b4507cf48bca517f41f0feebd78/roles/ef689f96442945e8a7bc8cbfee13d909 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:21:25.942764 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6c5730-6c95-42a6-9885-4aca51f1b291 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:25.943289 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6c5730-6c95-42a6-9885-4aca51f1b291 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:25.944068 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6c5730-6c95-42a6-9885-4aca51f1b291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:25.958840 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 962/1920] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:21:25 2022] DELETE /identity/v3/users/de2dff9816a943afb842a41d467d8019 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:21:25.963734 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaee386-a782-463d-95ea-d4c890deccbc None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:25.964274 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaee386-a782-463d-95ea-d4c890deccbc None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:25.964714 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaee386-a782-463d-95ea-d4c890deccbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:25.969877 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-7f6c5730-6c95-42a6-9885-4aca51f1b291 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:21:25.971114 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-7f6c5730-6c95-42a6-9885-4aca51f1b291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:25.988509 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-ccaee386-a782-463d-95ea-d4c890deccbc None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:21:25.988795 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:25.990172 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-ccaee386-a782-463d-95ea-d4c890deccbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:26.075589 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 963/1921] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:21:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:21:26.076974 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 959/1922] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:21:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:21:26.085658 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaee386-a782-463d-95ea-d4c890deccbc None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:26.087453 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-ccaee386-a782-463d-95ea-d4c890deccbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:26.090984 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6c5730-6c95-42a6-9885-4aca51f1b291 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:26.094205 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-7f6c5730-6c95-42a6-9885-4aca51f1b291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:26.165017 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaee386-a782-463d-95ea-d4c890deccbc None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:26.169945 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f039a1f7-5cd3-4f84-9cd1-0edc42325905 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:26.170650 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f039a1f7-5cd3-4f84-9cd1-0edc42325905 service neutron] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:26.171678 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f039a1f7-5cd3-4f84-9cd1-0edc42325905 service neutron] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:26.172031 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f039a1f7-5cd3-4f84-9cd1-0edc42325905 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:26.184212 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f039a1f7-5cd3-4f84-9cd1-0edc42325905 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:26.185707 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f039a1f7-5cd3-4f84-9cd1-0edc42325905 service neutron] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:26.188158 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6c5730-6c95-42a6-9885-4aca51f1b291 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:26.192417 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6956e4df-83a3-429d-a8c7-6fc1edc31471 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:26.195435 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6956e4df-83a3-429d-a8c7-6fc1edc31471 service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:26.195435 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6956e4df-83a3-429d-a8c7-6fc1edc31471 service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:26.195435 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6956e4df-83a3-429d-a8c7-6fc1edc31471 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:26.203701 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 964/1923] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:21:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:21:26.205037 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6956e4df-83a3-429d-a8c7-6fc1edc31471 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:26.206551 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6956e4df-83a3-429d-a8c7-6fc1edc31471 service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:26.222714 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 960/1924] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:21:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 00:21:26.870996 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:27.107905 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f039a1f7-5cd3-4f84-9cd1-0edc42325905 service neutron] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:27.109346 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-f039a1f7-5cd3-4f84-9cd1-0edc42325905 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:27.122722 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f039a1f7-5cd3-4f84-9cd1-0edc42325905 service neutron] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:27.128521 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0214d53-4b2d-4962-805f-d5ac1327520d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:27.129163 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0214d53-4b2d-4962-805f-d5ac1327520d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:27.129593 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0214d53-4b2d-4962-805f-d5ac1327520d admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:27.130069 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0214d53-4b2d-4962-805f-d5ac1327520d admin admin] PATH_INFO: `/v3/projects/ac3fc5ff2b464891ba5fd114cea03198` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:27.131459 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0214d53-4b2d-4962-805f-d5ac1327520d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac3fc5ff2b464891ba5fd114cea03198)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:27.133517 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0214d53-4b2d-4962-805f-d5ac1327520d admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:27.295811 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-d0214d53-4b2d-4962-805f-d5ac1327520d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac3fc5ff2b464891ba5fd114cea03198', 'request_id': 'req-d0214d53-4b2d-4962-805f-d5ac1327520d'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:27.307493 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-d0214d53-4b2d-4962-805f-d5ac1327520d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac3fc5ff2b464891ba5fd114cea03198', 'request_id': 'req-d0214d53-4b2d-4962-805f-d5ac1327520d'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:27.389141 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-d0214d53-4b2d-4962-805f-d5ac1327520d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac3fc5ff2b464891ba5fd114cea03198', 'request_id': 'req-d0214d53-4b2d-4962-805f-d5ac1327520d'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:27.402395 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-d0214d53-4b2d-4962-805f-d5ac1327520d admin admin] The token cache is being invalidate because project ac3fc5ff2b464891ba5fd114cea03198 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81747) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:21:27.402688 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-d0214d53-4b2d-4962-805f-d5ac1327520d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:27.403394 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-d0214d53-4b2d-4962-805f-d5ac1327520d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:27.404925 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 965/1925] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:21:27 2022] DELETE /identity/v3/projects/ac3fc5ff2b464891ba5fd114cea03198 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:21:27.996355 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6687296e-d6d9-4cd2-aec8-eba7605e582c None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:27.997745 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6687296e-d6d9-4cd2-aec8-eba7605e582c None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:27.998129 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6687296e-d6d9-4cd2-aec8-eba7605e582c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:28.058399 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-6687296e-d6d9-4cd2-aec8-eba7605e582c None None] MFA Rules not processed for user `b0791b4507cf48bca517f41f0feebd78`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:21:28.059806 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-6687296e-d6d9-4cd2-aec8-eba7605e582c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:28.186550 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f61a6e1-611f-416a-bd19-9eb00b08a624 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:28.188243 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f61a6e1-611f-416a-bd19-9eb00b08a624 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:28.189306 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f61a6e1-611f-416a-bd19-9eb00b08a624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:28.239514 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 961/1926] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:21:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:21:28.263713 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-2f61a6e1-611f-416a-bd19-9eb00b08a624 None None] MFA Rules not processed for user `63474c55edbf4b22ad609471a2414ede`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:21:28.265491 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-2f61a6e1-611f-416a-bd19-9eb00b08a624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:28.284480 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:28.294336 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6687296e-d6d9-4cd2-aec8-eba7605e582c None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:28.299851 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-6687296e-d6d9-4cd2-aec8-eba7605e582c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:28.377598 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6687296e-d6d9-4cd2-aec8-eba7605e582c None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:28.382939 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8435db47-3922-4962-8152-ab235c199caf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:28.383962 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8435db47-3922-4962-8152-ab235c199caf service cinder] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:28.384570 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8435db47-3922-4962-8152-ab235c199caf service cinder] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:28.385692 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8435db47-3922-4962-8152-ab235c199caf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:28.408659 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8435db47-3922-4962-8152-ab235c199caf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:28.408659 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8435db47-3922-4962-8152-ab235c199caf service cinder] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:28.411263 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 966/1927] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:21:28 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:21:28.423925 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 962/1928] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:21:28 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:21:28.426853 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f61a6e1-611f-416a-bd19-9eb00b08a624 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:28.428345 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-2f61a6e1-611f-416a-bd19-9eb00b08a624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:28.546865 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f61a6e1-611f-416a-bd19-9eb00b08a624 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:28.553200 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16efaf9-fbf5-44f2-9b51-6d9cadc75095 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a515b75aa3944912ad2364ffd7924f40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:28.553840 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16efaf9-fbf5-44f2-9b51-6d9cadc75095 service nova] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:28.554115 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16efaf9-fbf5-44f2-9b51-6d9cadc75095 service nova] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:28.554378 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16efaf9-fbf5-44f2-9b51-6d9cadc75095 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:28.563775 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16efaf9-fbf5-44f2-9b51-6d9cadc75095 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:28.565361 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16efaf9-fbf5-44f2-9b51-6d9cadc75095 service nova] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:28.575268 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 967/1929] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:21:28 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:21:29.477074 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:29.909609 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:INSERT=4,keystone:SELECT=86,keystone:DELETE=10 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:30.109275 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:31.066054 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ed2d93-01f3-42a8-9e14-98956d2209da None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:31.066562 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ed2d93-01f3-42a8-9e14-98956d2209da None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:31.066692 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ed2d93-01f3-42a8-9e14-98956d2209da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:31.086782 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-a2ed2d93-01f3-42a8-9e14-98956d2209da None None] MFA Rules not processed for user `63474c55edbf4b22ad609471a2414ede`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:21:31.088495 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-a2ed2d93-01f3-42a8-9e14-98956d2209da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:31.095795 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 963/1930] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:21:31 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 00:21:31.383624 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:31.532218 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:32.293464 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:32.343297 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39629434-4f55-4725-8881-41d3a54f9b01 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:32.343601 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39629434-4f55-4725-8881-41d3a54f9b01 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:32.343877 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39629434-4f55-4725-8881-41d3a54f9b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:32.363766 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-39629434-4f55-4725-8881-41d3a54f9b01 None None] MFA Rules not processed for user `b0791b4507cf48bca517f41f0feebd78`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:21:32.365079 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-39629434-4f55-4725-8881-41d3a54f9b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:32.372019 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 968/1931] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:21:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:21:32.641572 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:32.808985 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ed2d93-01f3-42a8-9e14-98956d2209da None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:32.810714 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-a2ed2d93-01f3-42a8-9e14-98956d2209da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:32.881788 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ed2d93-01f3-42a8-9e14-98956d2209da None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:32.887018 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe50d17-b003-4bbd-8d04-4bd00a524f66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:32.887686 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe50d17-b003-4bbd-8d04-4bd00a524f66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:32.888230 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe50d17-b003-4bbd-8d04-4bd00a524f66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:32.888501 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe50d17-b003-4bbd-8d04-4bd00a524f66 admin admin] PATH_INFO: `/v3/users/1bdc756abcc44428aaf1b2093cd9864d` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:32.889644 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe50d17-b003-4bbd-8d04-4bd00a524f66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bdc756abcc44428aaf1b2093cd9864d)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:32.891691 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe50d17-b003-4bbd-8d04-4bd00a524f66 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:33.151894 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-bfe50d17-b003-4bbd-8d04-4bd00a524f66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bdc756abcc44428aaf1b2093cd9864d'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:33.153733 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-bfe50d17-b003-4bbd-8d04-4bd00a524f66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bdc756abcc44428aaf1b2093cd9864d'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:33.199835 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-bfe50d17-b003-4bbd-8d04-4bd00a524f66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bdc756abcc44428aaf1b2093cd9864d'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:33.343095 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-bfe50d17-b003-4bbd-8d04-4bd00a524f66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bdc756abcc44428aaf1b2093cd9864d'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:33.344074 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-bfe50d17-b003-4bbd-8d04-4bd00a524f66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bdc756abcc44428aaf1b2093cd9864d'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:33.358889 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 964/1932] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:21:32 2022] DELETE /identity/v3/users/1bdc756abcc44428aaf1b2093cd9864d => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 00:21:33.363330 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed1d21f-7bf9-48c1-91d1-038c8685e0a4 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:33.363916 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed1d21f-7bf9-48c1-91d1-038c8685e0a4 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:33.363916 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed1d21f-7bf9-48c1-91d1-038c8685e0a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:33.384324 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-7ed1d21f-7bf9-48c1-91d1-038c8685e0a4 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:21:33.386476 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-7ed1d21f-7bf9-48c1-91d1-038c8685e0a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:33.450269 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 969/1933] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:21:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:21:33.460321 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe50d17-b003-4bbd-8d04-4bd00a524f66 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:33.462107 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-bfe50d17-b003-4bbd-8d04-4bd00a524f66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:33.525464 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe50d17-b003-4bbd-8d04-4bd00a524f66 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:33.529182 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6c369d-2247-4f91-98be-ce0a837514f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:33.529791 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c369d-2247-4f91-98be-ce0a837514f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:33.530067 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c369d-2247-4f91-98be-ce0a837514f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:33.530334 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c369d-2247-4f91-98be-ce0a837514f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:33.539761 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6c369d-2247-4f91-98be-ce0a837514f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:33.542139 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6c369d-2247-4f91-98be-ce0a837514f0 service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:33.553759 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 965/1934] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:21:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:21:33.661944 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:33.995674 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed1d21f-7bf9-48c1-91d1-038c8685e0a4 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:33.998018 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-7ed1d21f-7bf9-48c1-91d1-038c8685e0a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:34.010703 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed1d21f-7bf9-48c1-91d1-038c8685e0a4 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:34.015912 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ebb7a4-e5d4-4f2f-ae3a-e33574f9feeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:34.016677 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ebb7a4-e5d4-4f2f-ae3a-e33574f9feeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:34.016957 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ebb7a4-e5d4-4f2f-ae3a-e33574f9feeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:34.017225 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ebb7a4-e5d4-4f2f-ae3a-e33574f9feeb admin admin] PATH_INFO: `/v3/projects/61a665795db4405db2f2602f7a6c1cfe` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:34.018573 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ebb7a4-e5d4-4f2f-ae3a-e33574f9feeb admin admin] RBAC: Authorizing `identity:delete_project(project_id=61a665795db4405db2f2602f7a6c1cfe)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:34.022431 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ebb7a4-e5d4-4f2f-ae3a-e33574f9feeb admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:34.148043 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-26ebb7a4-e5d4-4f2f-ae3a-e33574f9feeb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61a665795db4405db2f2602f7a6c1cfe', 'request_id': 'req-26ebb7a4-e5d4-4f2f-ae3a-e33574f9feeb'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:34.153910 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:34.161125 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-26ebb7a4-e5d4-4f2f-ae3a-e33574f9feeb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61a665795db4405db2f2602f7a6c1cfe', 'request_id': 'req-26ebb7a4-e5d4-4f2f-ae3a-e33574f9feeb'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:34.277614 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-26ebb7a4-e5d4-4f2f-ae3a-e33574f9feeb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61a665795db4405db2f2602f7a6c1cfe', 'request_id': 'req-26ebb7a4-e5d4-4f2f-ae3a-e33574f9feeb'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:34.290174 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-26ebb7a4-e5d4-4f2f-ae3a-e33574f9feeb admin admin] The token cache is being invalidate because project 61a665795db4405db2f2602f7a6c1cfe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81747) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:21:34.290458 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-26ebb7a4-e5d4-4f2f-ae3a-e33574f9feeb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:34.291310 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-26ebb7a4-e5d4-4f2f-ae3a-e33574f9feeb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:34.293167 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 970/1935] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:21:33 2022] DELETE /identity/v3/projects/61a665795db4405db2f2602f7a6c1cfe => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:21:34.407002 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:35.305795 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:35.416601 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:35.872086 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:35.946325 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:36.441728 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=81,keystone:INSERT=5,keystone:DELETE=14 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:36.685621 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6c369d-2247-4f91-98be-ce0a837514f0 service neutron] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:36.687300 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-2a6c369d-2247-4f91-98be-ce0a837514f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:36.700592 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6c369d-2247-4f91-98be-ce0a837514f0 service neutron] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:36.742272 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:36.775347 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b56f62-4cd0-422d-866a-369f88472ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:36.775945 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b56f62-4cd0-422d-866a-369f88472ca1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:36.776234 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b56f62-4cd0-422d-866a-369f88472ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:36.776517 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b56f62-4cd0-422d-866a-369f88472ca1 admin admin] PATH_INFO: `/v3/users/f5a57ad052d746b28c87030a10f3f8ed` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:36.777704 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b56f62-4cd0-422d-866a-369f88472ca1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5a57ad052d746b28c87030a10f3f8ed)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:36.779733 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b56f62-4cd0-422d-866a-369f88472ca1 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:37.057737 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-98b56f62-4cd0-422d-866a-369f88472ca1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5a57ad052d746b28c87030a10f3f8ed'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:37.058629 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-98b56f62-4cd0-422d-866a-369f88472ca1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5a57ad052d746b28c87030a10f3f8ed'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:37.072986 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-98b56f62-4cd0-422d-866a-369f88472ca1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5a57ad052d746b28c87030a10f3f8ed'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:37.160950 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-98b56f62-4cd0-422d-866a-369f88472ca1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5a57ad052d746b28c87030a10f3f8ed'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:37.162430 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-98b56f62-4cd0-422d-866a-369f88472ca1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5a57ad052d746b28c87030a10f3f8ed'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:37.178174 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 966/1936] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:21:36 2022] DELETE /identity/v3/users/f5a57ad052d746b28c87030a10f3f8ed => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:21:37.182718 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da33a361-e0bc-4294-8be1-89219b8d48fc None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:37.182996 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da33a361-e0bc-4294-8be1-89219b8d48fc None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:37.183263 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da33a361-e0bc-4294-8be1-89219b8d48fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:37.205354 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-da33a361-e0bc-4294-8be1-89219b8d48fc None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:21:37.206733 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-da33a361-e0bc-4294-8be1-89219b8d48fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:37.280058 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 971/1937] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:21:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:21:37.293376 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b56f62-4cd0-422d-866a-369f88472ca1 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:37.295115 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-98b56f62-4cd0-422d-866a-369f88472ca1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:37.358265 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b56f62-4cd0-422d-866a-369f88472ca1 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:37.362012 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171ff8b1-7ea5-41ee-b141-9c001ff26a85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:37.363257 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171ff8b1-7ea5-41ee-b141-9c001ff26a85 service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:37.363709 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171ff8b1-7ea5-41ee-b141-9c001ff26a85 service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:37.364088 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171ff8b1-7ea5-41ee-b141-9c001ff26a85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:37.374307 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171ff8b1-7ea5-41ee-b141-9c001ff26a85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:37.375947 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171ff8b1-7ea5-41ee-b141-9c001ff26a85 service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:37.386915 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 967/1938] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:21:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:21:37.424127 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=91,keystone:INSERT=1,keystone:DELETE=8 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:38.006181 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:38.104655 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da33a361-e0bc-4294-8be1-89219b8d48fc None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:38.106209 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-da33a361-e0bc-4294-8be1-89219b8d48fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:38.118378 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da33a361-e0bc-4294-8be1-89219b8d48fc None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:38.123758 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba72c3a-8b39-4f10-b7f6-97770289cd21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:38.124651 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba72c3a-8b39-4f10-b7f6-97770289cd21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:38.125088 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba72c3a-8b39-4f10-b7f6-97770289cd21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:38.125525 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba72c3a-8b39-4f10-b7f6-97770289cd21 admin admin] PATH_INFO: `/v3/projects/5a073bf9f0974dcdba242b6fce21e71e` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:38.126848 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba72c3a-8b39-4f10-b7f6-97770289cd21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a073bf9f0974dcdba242b6fce21e71e)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:38.129327 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba72c3a-8b39-4f10-b7f6-97770289cd21 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:38.265478 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-bba72c3a-8b39-4f10-b7f6-97770289cd21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a073bf9f0974dcdba242b6fce21e71e', 'request_id': 'req-bba72c3a-8b39-4f10-b7f6-97770289cd21'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:38.275079 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-bba72c3a-8b39-4f10-b7f6-97770289cd21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a073bf9f0974dcdba242b6fce21e71e', 'request_id': 'req-bba72c3a-8b39-4f10-b7f6-97770289cd21'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:38.363189 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-bba72c3a-8b39-4f10-b7f6-97770289cd21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a073bf9f0974dcdba242b6fce21e71e', 'request_id': 'req-bba72c3a-8b39-4f10-b7f6-97770289cd21'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:38.375165 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-bba72c3a-8b39-4f10-b7f6-97770289cd21 admin admin] The token cache is being invalidate because project 5a073bf9f0974dcdba242b6fce21e71e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81747) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:21:38.375483 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-bba72c3a-8b39-4f10-b7f6-97770289cd21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:38.376417 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-bba72c3a-8b39-4f10-b7f6-97770289cd21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:38.378181 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 972/1939] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:21:38 2022] DELETE /identity/v3/projects/5a073bf9f0974dcdba242b6fce21e71e => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:21:38.383108 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abaa31c-1992-48a5-ac6e-ae28c860bf25 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:38.383659 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abaa31c-1992-48a5-ac6e-ae28c860bf25 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:38.383659 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abaa31c-1992-48a5-ac6e-ae28c860bf25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:38.405123 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-2abaa31c-1992-48a5-ac6e-ae28c860bf25 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:21:38.406516 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-2abaa31c-1992-48a5-ac6e-ae28c860bf25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:38.483835 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 968/1940] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:21:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:21:38.484192 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00b925-d169-4752-abfa-70684a867421 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:38.484482 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00b925-d169-4752-abfa-70684a867421 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:38.484773 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00b925-d169-4752-abfa-70684a867421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:38.505243 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-8b00b925-d169-4752-abfa-70684a867421 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:21:38.511985 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-8b00b925-d169-4752-abfa-70684a867421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:38.514883 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 973/1941] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:21:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 00:21:38.519092 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abaa31c-1992-48a5-ac6e-ae28c860bf25 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:38.529244 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abaa31c-1992-48a5-ac6e-ae28c860bf25 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:38.533813 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16050f7d-421d-49ce-92e8-e0e7f2a88010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:38.534412 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16050f7d-421d-49ce-92e8-e0e7f2a88010 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:38.534690 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16050f7d-421d-49ce-92e8-e0e7f2a88010 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:38.534961 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16050f7d-421d-49ce-92e8-e0e7f2a88010 admin admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:38.535636 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16050f7d-421d-49ce-92e8-e0e7f2a88010 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:21:38.535938 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16050f7d-421d-49ce-92e8-e0e7f2a88010 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:38.537541 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16050f7d-421d-49ce-92e8-e0e7f2a88010 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:38.552690 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 969/1942] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Oct 12 00:21:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:21:38.556860 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b00b925-d169-4752-abfa-70684a867421 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:38.566292 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b00b925-d169-4752-abfa-70684a867421 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:38.571098 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd999aea-19a8-4f11-8a7a-8a88382e0ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:38.572058 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd999aea-19a8-4f11-8a7a-8a88382e0ead admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:38.572338 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd999aea-19a8-4f11-8a7a-8a88382e0ead admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:38.572790 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd999aea-19a8-4f11-8a7a-8a88382e0ead admin admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:38.573436 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd999aea-19a8-4f11-8a7a-8a88382e0ead admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:38.575578 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd999aea-19a8-4f11-8a7a-8a88382e0ead admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:38.668505 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 974/1943] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:21:38 2022] POST /identity/v3/projects => generated 383 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:21:38.669888 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16050f7d-421d-49ce-92e8-e0e7f2a88010 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:38.683499 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16050f7d-421d-49ce-92e8-e0e7f2a88010 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:38.743915 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d304ef7-30e5-45f2-8146-343a3359769f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:38.744531 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d304ef7-30e5-45f2-8146-343a3359769f admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:38.744809 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d304ef7-30e5-45f2-8146-343a3359769f admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:38.745074 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d304ef7-30e5-45f2-8146-343a3359769f admin admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:38.745704 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d304ef7-30e5-45f2-8146-343a3359769f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:38.747713 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d304ef7-30e5-45f2-8146-343a3359769f admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:38.844088 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 970/1944] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:21:38 2022] POST /identity/v3/users => generated 292 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:21:38.848641 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd999aea-19a8-4f11-8a7a-8a88382e0ead admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:38.859481 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd999aea-19a8-4f11-8a7a-8a88382e0ead admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:38.864694 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8e5f43-9514-4851-940b-2c4a97fac2af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:38.864948 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8e5f43-9514-4851-940b-2c4a97fac2af admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:38.865757 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8e5f43-9514-4851-940b-2c4a97fac2af admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:38.866199 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8e5f43-9514-4851-940b-2c4a97fac2af admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:38.866847 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8e5f43-9514-4851-940b-2c4a97fac2af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:38.868603 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8e5f43-9514-4851-940b-2c4a97fac2af admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:38.878955 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 975/1945] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:21:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:21:38.883473 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d304ef7-30e5-45f2-8146-343a3359769f admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:38.894548 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d304ef7-30e5-45f2-8146-343a3359769f admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:38.899529 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb090ad-023c-4dc2-af0b-9e7d847c1a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:38.901318 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb090ad-023c-4dc2-af0b-9e7d847c1a0e admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:38.901604 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb090ad-023c-4dc2-af0b-9e7d847c1a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:38.901879 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb090ad-023c-4dc2-af0b-9e7d847c1a0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:38.902525 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb090ad-023c-4dc2-af0b-9e7d847c1a0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:38.904185 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb090ad-023c-4dc2-af0b-9e7d847c1a0e admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:38.915740 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 971/1946] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:21:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:21:38.919926 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8e5f43-9514-4851-940b-2c4a97fac2af admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:38.930122 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8e5f43-9514-4851-940b-2c4a97fac2af admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:38.935497 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b52690-7147-4d3f-a2ca-68380ff5d291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:38.936355 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b52690-7147-4d3f-a2ca-68380ff5d291 admin admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:38.936794 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b52690-7147-4d3f-a2ca-68380ff5d291 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:38.937242 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b52690-7147-4d3f-a2ca-68380ff5d291 admin admin] PATH_INFO: `/v3/projects/a913b713f2df4997bf07e8aaf120fe54/users/fb3e8f8ffbe443ac934fa880b513eece/roles/ef689f96442945e8a7bc8cbfee13d909` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:38.953852 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b52690-7147-4d3f-a2ca-68380ff5d291 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a913b713f2df4997bf07e8aaf120fe54, user_id=fb3e8f8ffbe443ac934fa880b513eece, role_id=ef689f96442945e8a7bc8cbfee13d909)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:38.957434 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b52690-7147-4d3f-a2ca-68380ff5d291 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:38.960649 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:39.034788 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 976/1947] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:21:38 2022] PUT /identity/v3/projects/a913b713f2df4997bf07e8aaf120fe54/users/fb3e8f8ffbe443ac934fa880b513eece/roles/ef689f96442945e8a7bc8cbfee13d909 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:21:39.050502 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72424427-e02a-4aaa-9609-4bd7ca113953 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:39.050784 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72424427-e02a-4aaa-9609-4bd7ca113953 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:39.051083 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72424427-e02a-4aaa-9609-4bd7ca113953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:39.078894 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-72424427-e02a-4aaa-9609-4bd7ca113953 None None] MFA Rules not processed for user `fb3e8f8ffbe443ac934fa880b513eece`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:21:39.080142 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-72424427-e02a-4aaa-9609-4bd7ca113953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:39.143767 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 972/1948] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:21:39 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:21:39.154870 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b52690-7147-4d3f-a2ca-68380ff5d291 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:39.156574 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-21b52690-7147-4d3f-a2ca-68380ff5d291 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:39.167696 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:39.215209 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b52690-7147-4d3f-a2ca-68380ff5d291 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:39.218864 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3093cc43-22df-44cb-b650-44460f29250e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:39.219516 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3093cc43-22df-44cb-b650-44460f29250e service cinder] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:39.219793 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3093cc43-22df-44cb-b650-44460f29250e service cinder] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:39.220064 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3093cc43-22df-44cb-b650-44460f29250e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:39.228564 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3093cc43-22df-44cb-b650-44460f29250e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:39.232308 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3093cc43-22df-44cb-b650-44460f29250e service cinder] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:39.238816 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 977/1949] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:21:39 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:21:39.617969 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:39.740865 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:40.003289 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:40.032384 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:40.317538 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:40.377542 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:40.491347 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:40.522761 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:40.914603 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:40.971573 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:41.131813 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:41.160425 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:41.425789 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:41.503021 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:41.531664 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:41.803327 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:41.886882 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:42.117460 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:42.185803 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:42.334335 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:42.357770 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72424427-e02a-4aaa-9609-4bd7ca113953 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:42.418862 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72424427-e02a-4aaa-9609-4bd7ca113953 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:42.423388 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b5ecd1-d681-4ac0-bd2f-2bb148ad836b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:42.424030 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b5ecd1-d681-4ac0-bd2f-2bb148ad836b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:42.424308 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b5ecd1-d681-4ac0-bd2f-2bb148ad836b admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:42.424581 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b5ecd1-d681-4ac0-bd2f-2bb148ad836b admin admin] PATH_INFO: `/v3/users/fb3e8f8ffbe443ac934fa880b513eece` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:42.425697 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b5ecd1-d681-4ac0-bd2f-2bb148ad836b admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb3e8f8ffbe443ac934fa880b513eece)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:42.427695 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b5ecd1-d681-4ac0-bd2f-2bb148ad836b admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:42.599240 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-f5b5ecd1-d681-4ac0-bd2f-2bb148ad836b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb3e8f8ffbe443ac934fa880b513eece'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:42.600153 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-f5b5ecd1-d681-4ac0-bd2f-2bb148ad836b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb3e8f8ffbe443ac934fa880b513eece'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:42.612962 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-f5b5ecd1-d681-4ac0-bd2f-2bb148ad836b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb3e8f8ffbe443ac934fa880b513eece'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:42.671641 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-f5b5ecd1-d681-4ac0-bd2f-2bb148ad836b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb3e8f8ffbe443ac934fa880b513eece'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:42.672291 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-f5b5ecd1-d681-4ac0-bd2f-2bb148ad836b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb3e8f8ffbe443ac934fa880b513eece'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:42.686202 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 973/1950] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:21:42 2022] DELETE /identity/v3/users/fb3e8f8ffbe443ac934fa880b513eece => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 00:21:42.690540 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766bd4eb-2476-48d8-9061-ab062da0bdab None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:42.691018 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766bd4eb-2476-48d8-9061-ab062da0bdab None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:42.691138 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766bd4eb-2476-48d8-9061-ab062da0bdab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:42.695278 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:42.713262 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-766bd4eb-2476-48d8-9061-ab062da0bdab None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:21:42.714578 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-766bd4eb-2476-48d8-9061-ab062da0bdab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:42.755889 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:42.777198 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 978/1951] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:21:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 00:21:42.786137 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b5ecd1-d681-4ac0-bd2f-2bb148ad836b admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:42.787591 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-f5b5ecd1-d681-4ac0-bd2f-2bb148ad836b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:42.854788 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b5ecd1-d681-4ac0-bd2f-2bb148ad836b admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:42.858513 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f328a48a-be2b-440c-b8e1-067709198d28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:42.859126 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f328a48a-be2b-440c-b8e1-067709198d28 service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:42.859420 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f328a48a-be2b-440c-b8e1-067709198d28 service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:42.859699 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f328a48a-be2b-440c-b8e1-067709198d28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:42.871755 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f328a48a-be2b-440c-b8e1-067709198d28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:42.873893 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f328a48a-be2b-440c-b8e1-067709198d28 service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:42.885421 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 974/1952] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:21:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:21:43.381140 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766bd4eb-2476-48d8-9061-ab062da0bdab None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:43.384903 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-766bd4eb-2476-48d8-9061-ab062da0bdab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:43.397018 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766bd4eb-2476-48d8-9061-ab062da0bdab None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:43.404057 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8d0664-4c72-4c36-b66f-34d3e15e481f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:43.404878 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8d0664-4c72-4c36-b66f-34d3e15e481f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:43.405145 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8d0664-4c72-4c36-b66f-34d3e15e481f admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:43.405421 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8d0664-4c72-4c36-b66f-34d3e15e481f admin admin] PATH_INFO: `/v3/projects/a913b713f2df4997bf07e8aaf120fe54` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:43.406569 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8d0664-4c72-4c36-b66f-34d3e15e481f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a913b713f2df4997bf07e8aaf120fe54)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:43.408523 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8d0664-4c72-4c36-b66f-34d3e15e481f admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:43.527046 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-ec8d0664-4c72-4c36-b66f-34d3e15e481f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a913b713f2df4997bf07e8aaf120fe54', 'request_id': 'req-ec8d0664-4c72-4c36-b66f-34d3e15e481f'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:43.537928 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-ec8d0664-4c72-4c36-b66f-34d3e15e481f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a913b713f2df4997bf07e8aaf120fe54', 'request_id': 'req-ec8d0664-4c72-4c36-b66f-34d3e15e481f'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:43.657454 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-ec8d0664-4c72-4c36-b66f-34d3e15e481f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a913b713f2df4997bf07e8aaf120fe54', 'request_id': 'req-ec8d0664-4c72-4c36-b66f-34d3e15e481f'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:43.668927 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-ec8d0664-4c72-4c36-b66f-34d3e15e481f admin admin] The token cache is being invalidate because project a913b713f2df4997bf07e8aaf120fe54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81747) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:21:43.669209 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-ec8d0664-4c72-4c36-b66f-34d3e15e481f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:43.669837 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-ec8d0664-4c72-4c36-b66f-34d3e15e481f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:43.671507 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 979/1953] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:21:43 2022] DELETE /identity/v3/projects/a913b713f2df4997bf07e8aaf120fe54 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:21:43.676302 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9569f495-deb8-430a-9298-f0f62058da78 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:43.676766 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9569f495-deb8-430a-9298-f0f62058da78 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:43.676853 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9569f495-deb8-430a-9298-f0f62058da78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:43.697526 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-9569f495-deb8-430a-9298-f0f62058da78 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:21:43.698732 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-9569f495-deb8-430a-9298-f0f62058da78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:43.761600 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 975/1954] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:21:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 00:21:43.769953 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c5a839-f3a5-4016-b0ac-f10b65f88f6b None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:43.770425 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c5a839-f3a5-4016-b0ac-f10b65f88f6b None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:43.770515 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c5a839-f3a5-4016-b0ac-f10b65f88f6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:43.790748 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-a1c5a839-f3a5-4016-b0ac-f10b65f88f6b None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:21:43.795889 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-a1c5a839-f3a5-4016-b0ac-f10b65f88f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:43.800246 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 980/1955] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:21:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:21:43.804532 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9569f495-deb8-430a-9298-f0f62058da78 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:43.816495 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9569f495-deb8-430a-9298-f0f62058da78 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:43.822287 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78adba59-6707-429a-84d0-c060a7703427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:43.822904 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78adba59-6707-429a-84d0-c060a7703427 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:43.823355 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78adba59-6707-429a-84d0-c060a7703427 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:43.823564 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78adba59-6707-429a-84d0-c060a7703427 admin admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:43.828523 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78adba59-6707-429a-84d0-c060a7703427 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:21:43.828523 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78adba59-6707-429a-84d0-c060a7703427 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:43.828523 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78adba59-6707-429a-84d0-c060a7703427 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:43.838475 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 976/1956] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Oct 12 00:21:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:21:43.842759 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c5a839-f3a5-4016-b0ac-f10b65f88f6b None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:43.854655 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c5a839-f3a5-4016-b0ac-f10b65f88f6b None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:43.860446 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4677d288-216f-4aa1-819a-0bf955f8c991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:43.861064 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4677d288-216f-4aa1-819a-0bf955f8c991 admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:43.861341 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4677d288-216f-4aa1-819a-0bf955f8c991 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:43.861612 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4677d288-216f-4aa1-819a-0bf955f8c991 admin admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:43.862415 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4677d288-216f-4aa1-819a-0bf955f8c991 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:43.864447 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4677d288-216f-4aa1-819a-0bf955f8c991 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:43.966149 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 981/1957] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:21:43 2022] POST /identity/v3/projects => generated 375 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:21:43.970011 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78adba59-6707-429a-84d0-c060a7703427 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:43.991898 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78adba59-6707-429a-84d0-c060a7703427 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:44.051204 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c21d58-344a-4923-a3d4-aa11ea559acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:44.051864 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c21d58-344a-4923-a3d4-aa11ea559acf admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:44.052142 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c21d58-344a-4923-a3d4-aa11ea559acf admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:44.052412 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c21d58-344a-4923-a3d4-aa11ea559acf admin admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:44.053051 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c21d58-344a-4923-a3d4-aa11ea559acf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:44.054982 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c21d58-344a-4923-a3d4-aa11ea559acf admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:44.137386 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:44.144354 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 977/1958] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:21:43 2022] POST /identity/v3/users => generated 288 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:21:44.148373 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4677d288-216f-4aa1-819a-0bf955f8c991 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:44.159858 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4677d288-216f-4aa1-819a-0bf955f8c991 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:44.165992 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e36362-555b-4565-8919-005d6620bbfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:44.166955 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e36362-555b-4565-8919-005d6620bbfc admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:44.168492 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e36362-555b-4565-8919-005d6620bbfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:44.168768 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e36362-555b-4565-8919-005d6620bbfc admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:44.169432 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e36362-555b-4565-8919-005d6620bbfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:44.171121 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e36362-555b-4565-8919-005d6620bbfc admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:44.186013 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 982/1959] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:21:44 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:21:44.186816 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c21d58-344a-4923-a3d4-aa11ea559acf admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:44.202696 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c21d58-344a-4923-a3d4-aa11ea559acf admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:44.209301 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69872105-fe18-43ba-8f0e-2539dd600fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:44.210947 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69872105-fe18-43ba-8f0e-2539dd600fbd admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:44.211236 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69872105-fe18-43ba-8f0e-2539dd600fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:44.211623 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69872105-fe18-43ba-8f0e-2539dd600fbd admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:44.212357 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69872105-fe18-43ba-8f0e-2539dd600fbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:44.214648 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69872105-fe18-43ba-8f0e-2539dd600fbd admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:44.229976 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 978/1960] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:21:44 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 00:21:44.234332 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e36362-555b-4565-8919-005d6620bbfc admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:44.248465 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e36362-555b-4565-8919-005d6620bbfc admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:44.253423 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af4d39a-4fee-4e38-8239-1d5de9563a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:44.254342 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af4d39a-4fee-4e38-8239-1d5de9563a10 admin admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:44.254691 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af4d39a-4fee-4e38-8239-1d5de9563a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:44.255030 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af4d39a-4fee-4e38-8239-1d5de9563a10 admin admin] PATH_INFO: `/v3/projects/1e1a6118eef1467abffb598652cd6ada/users/5cf85e1b70234f448776d4dbbd905c13/roles/ef689f96442945e8a7bc8cbfee13d909` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:44.276004 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af4d39a-4fee-4e38-8239-1d5de9563a10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e1a6118eef1467abffb598652cd6ada, user_id=5cf85e1b70234f448776d4dbbd905c13, role_id=ef689f96442945e8a7bc8cbfee13d909)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:44.278898 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af4d39a-4fee-4e38-8239-1d5de9563a10 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:44.356396 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 983/1961] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:21:44 2022] PUT /identity/v3/projects/1e1a6118eef1467abffb598652cd6ada/users/5cf85e1b70234f448776d4dbbd905c13/roles/ef689f96442945e8a7bc8cbfee13d909 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:21:44.368147 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a653162-aaec-4da4-a6f5-79cc03f7877b None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:44.368433 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a653162-aaec-4da4-a6f5-79cc03f7877b None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:44.368706 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a653162-aaec-4da4-a6f5-79cc03f7877b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:44.399930 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-5a653162-aaec-4da4-a6f5-79cc03f7877b None None] MFA Rules not processed for user `5cf85e1b70234f448776d4dbbd905c13`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:21:44.401206 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-5a653162-aaec-4da4-a6f5-79cc03f7877b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:44.471231 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 979/1962] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:21:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:21:44.484352 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af4d39a-4fee-4e38-8239-1d5de9563a10 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:44.486099 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-8af4d39a-4fee-4e38-8239-1d5de9563a10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:44.544605 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af4d39a-4fee-4e38-8239-1d5de9563a10 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:44.548273 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87df3375-b495-4be3-8468-ac03436d6412 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:44.548890 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87df3375-b495-4be3-8468-ac03436d6412 service cinder] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:44.549172 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87df3375-b495-4be3-8468-ac03436d6412 service cinder] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:44.549444 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87df3375-b495-4be3-8468-ac03436d6412 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:44.558051 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87df3375-b495-4be3-8468-ac03436d6412 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:44.559519 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87df3375-b495-4be3-8468-ac03436d6412 service cinder] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:44.568467 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 984/1963] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:21:44 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:21:54.564371 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:55.862836 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=38 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:56.254791 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a653162-aaec-4da4-a6f5-79cc03f7877b None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:56.324053 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a653162-aaec-4da4-a6f5-79cc03f7877b None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:56.328859 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a0954e-3261-4f20-a82e-dec9d09954c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:56.329588 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a0954e-3261-4f20-a82e-dec9d09954c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:56.329866 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a0954e-3261-4f20-a82e-dec9d09954c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:56.330230 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a0954e-3261-4f20-a82e-dec9d09954c6 admin admin] PATH_INFO: `/v3/users/5cf85e1b70234f448776d4dbbd905c13` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:56.331376 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a0954e-3261-4f20-a82e-dec9d09954c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cf85e1b70234f448776d4dbbd905c13)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:56.333664 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a0954e-3261-4f20-a82e-dec9d09954c6 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:56.487719 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-c2a0954e-3261-4f20-a82e-dec9d09954c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cf85e1b70234f448776d4dbbd905c13'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:56.488385 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-c2a0954e-3261-4f20-a82e-dec9d09954c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cf85e1b70234f448776d4dbbd905c13'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:56.502004 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-c2a0954e-3261-4f20-a82e-dec9d09954c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cf85e1b70234f448776d4dbbd905c13'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:56.571097 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-c2a0954e-3261-4f20-a82e-dec9d09954c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cf85e1b70234f448776d4dbbd905c13'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:56.571742 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-c2a0954e-3261-4f20-a82e-dec9d09954c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cf85e1b70234f448776d4dbbd905c13'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:56.586010 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 980/1964] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:21:56 2022] DELETE /identity/v3/users/5cf85e1b70234f448776d4dbbd905c13 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:21:56.590274 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612e2b55-704f-40b5-99be-0cb5b6dc976d None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:56.590735 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612e2b55-704f-40b5-99be-0cb5b6dc976d None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:56.590828 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612e2b55-704f-40b5-99be-0cb5b6dc976d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:56.611419 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-612e2b55-704f-40b5-99be-0cb5b6dc976d None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:21:56.613057 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-612e2b55-704f-40b5-99be-0cb5b6dc976d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:56.710819 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 985/1965] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:21:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:21:56.718799 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a0954e-3261-4f20-a82e-dec9d09954c6 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:56.720647 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-c2a0954e-3261-4f20-a82e-dec9d09954c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:56.783047 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a0954e-3261-4f20-a82e-dec9d09954c6 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:56.787254 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc09f49c-5ef9-4a4b-ad2f-e6472b6a401d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:56.787912 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc09f49c-5ef9-4a4b-ad2f-e6472b6a401d service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:56.788188 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc09f49c-5ef9-4a4b-ad2f-e6472b6a401d service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:56.788454 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc09f49c-5ef9-4a4b-ad2f-e6472b6a401d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:56.798411 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc09f49c-5ef9-4a4b-ad2f-e6472b6a401d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:56.800026 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc09f49c-5ef9-4a4b-ad2f-e6472b6a401d service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:56.811028 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 981/1966] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:21:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:21:57.252269 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bedeb20-22f6-416a-905f-f43f2129c063 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:57.252483 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bedeb20-22f6-416a-905f-f43f2129c063 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:57.252754 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bedeb20-22f6-416a-905f-f43f2129c063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:57.273190 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-1bedeb20-22f6-416a-905f-f43f2129c063 None None] MFA Rules not processed for user `de766cb582ba4c18b1fda9bf4ad43f65`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:21:57.274805 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-1bedeb20-22f6-416a-905f-f43f2129c063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:57.279942 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc09f49c-5ef9-4a4b-ad2f-e6472b6a401d service neutron] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:57.281479 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-fc09f49c-5ef9-4a4b-ad2f-e6472b6a401d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:57.306741 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc09f49c-5ef9-4a4b-ad2f-e6472b6a401d service neutron] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:57.312490 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257a38f1-9090-4af3-85b4-b84b38c0316e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:57.313133 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257a38f1-9090-4af3-85b4-b84b38c0316e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:57.313415 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257a38f1-9090-4af3-85b4-b84b38c0316e admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:57.313692 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257a38f1-9090-4af3-85b4-b84b38c0316e admin admin] PATH_INFO: `/v3/projects/1e1a6118eef1467abffb598652cd6ada` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:57.315579 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257a38f1-9090-4af3-85b4-b84b38c0316e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e1a6118eef1467abffb598652cd6ada)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:57.317578 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257a38f1-9090-4af3-85b4-b84b38c0316e admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:57.352948 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 986/1967] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:21:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:21:57.359748 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:57.465912 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-257a38f1-9090-4af3-85b4-b84b38c0316e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e1a6118eef1467abffb598652cd6ada', 'request_id': 'req-257a38f1-9090-4af3-85b4-b84b38c0316e'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:57.473765 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-257a38f1-9090-4af3-85b4-b84b38c0316e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e1a6118eef1467abffb598652cd6ada', 'request_id': 'req-257a38f1-9090-4af3-85b4-b84b38c0316e'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:57.552097 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-257a38f1-9090-4af3-85b4-b84b38c0316e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e1a6118eef1467abffb598652cd6ada', 'request_id': 'req-257a38f1-9090-4af3-85b4-b84b38c0316e'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:57.564260 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-257a38f1-9090-4af3-85b4-b84b38c0316e admin admin] The token cache is being invalidate because project 1e1a6118eef1467abffb598652cd6ada was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81746) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:21:57.564681 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-257a38f1-9090-4af3-85b4-b84b38c0316e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:57.565331 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-257a38f1-9090-4af3-85b4-b84b38c0316e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:21:57.566880 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 982/1968] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:21:57 2022] DELETE /identity/v3/projects/1e1a6118eef1467abffb598652cd6ada => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:21:57.572217 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30f438a-073a-4d14-892b-947660f6effb None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:57.573030 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30f438a-073a-4d14-892b-947660f6effb None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:57.573311 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30f438a-073a-4d14-892b-947660f6effb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:57.594741 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-c30f438a-073a-4d14-892b-947660f6effb None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:21:57.595990 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-c30f438a-073a-4d14-892b-947660f6effb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:57.669004 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 987/1969] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:21:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:21:57.676965 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff2b01-7195-4d67-973a-70530fe9ad5e None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:57.677327 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff2b01-7195-4d67-973a-70530fe9ad5e None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:57.677670 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff2b01-7195-4d67-973a-70530fe9ad5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:57.709020 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-02ff2b01-7195-4d67-973a-70530fe9ad5e None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:21:57.711598 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-02ff2b01-7195-4d67-973a-70530fe9ad5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:57.722719 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 983/1970] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:21:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:21:57.727651 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30f438a-073a-4d14-892b-947660f6effb None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:57.742048 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30f438a-073a-4d14-892b-947660f6effb None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:57.750335 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df73ce18-877b-4f8b-b857-e9c6e0172903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:57.751071 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df73ce18-877b-4f8b-b857-e9c6e0172903 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:57.751353 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df73ce18-877b-4f8b-b857-e9c6e0172903 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:57.751660 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df73ce18-877b-4f8b-b857-e9c6e0172903 admin admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:57.752326 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df73ce18-877b-4f8b-b857-e9c6e0172903 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:21:57.752623 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df73ce18-877b-4f8b-b857-e9c6e0172903 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:57.754433 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df73ce18-877b-4f8b-b857-e9c6e0172903 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:57.771257 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 988/1971] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Oct 12 00:21:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:21:57.777546 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ff2b01-7195-4d67-973a-70530fe9ad5e None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:57.787273 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ff2b01-7195-4d67-973a-70530fe9ad5e None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:57.795088 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e325aa7-41bc-43d1-98ee-7843c472c8be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:57.795858 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e325aa7-41bc-43d1-98ee-7843c472c8be admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:57.796243 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e325aa7-41bc-43d1-98ee-7843c472c8be admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:57.796620 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e325aa7-41bc-43d1-98ee-7843c472c8be admin admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:57.797376 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e325aa7-41bc-43d1-98ee-7843c472c8be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:57.799659 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e325aa7-41bc-43d1-98ee-7843c472c8be admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:57.911515 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 984/1972] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:21:57 2022] POST /identity/v3/projects => generated 385 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:21:57.912715 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df73ce18-877b-4f8b-b857-e9c6e0172903 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:57.923322 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df73ce18-877b-4f8b-b857-e9c6e0172903 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:57.964750 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:21:57.996119 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69e05f1-bc1f-40ed-b2ca-eee44f82a030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:57.997044 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69e05f1-bc1f-40ed-b2ca-eee44f82a030 admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:57.997324 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69e05f1-bc1f-40ed-b2ca-eee44f82a030 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:57.997595 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69e05f1-bc1f-40ed-b2ca-eee44f82a030 admin admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:57.998240 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69e05f1-bc1f-40ed-b2ca-eee44f82a030 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:58.000247 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69e05f1-bc1f-40ed-b2ca-eee44f82a030 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:58.080902 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 989/1973] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:21:57 2022] POST /identity/v3/users => generated 293 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:21:58.084988 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e325aa7-41bc-43d1-98ee-7843c472c8be admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:58.115798 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e325aa7-41bc-43d1-98ee-7843c472c8be admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:58.126234 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a808fd50-abae-49f9-b63f-517a799802c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:58.126915 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a808fd50-abae-49f9-b63f-517a799802c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:58.127219 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a808fd50-abae-49f9-b63f-517a799802c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:58.127730 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a808fd50-abae-49f9-b63f-517a799802c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:58.128271 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a808fd50-abae-49f9-b63f-517a799802c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:58.130359 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a808fd50-abae-49f9-b63f-517a799802c7 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:58.145865 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 985/1974] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:21:58 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 00:21:58.147466 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69e05f1-bc1f-40ed-b2ca-eee44f82a030 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:58.178892 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69e05f1-bc1f-40ed-b2ca-eee44f82a030 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:58.184615 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d144dcd0-c316-4e6c-947c-eeb6db1a6592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:58.185246 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d144dcd0-c316-4e6c-947c-eeb6db1a6592 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:58.185526 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d144dcd0-c316-4e6c-947c-eeb6db1a6592 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:58.185801 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d144dcd0-c316-4e6c-947c-eeb6db1a6592 admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:58.186446 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d144dcd0-c316-4e6c-947c-eeb6db1a6592 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:58.188143 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d144dcd0-c316-4e6c-947c-eeb6db1a6592 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:58.205023 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 990/1975] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:21:58 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:21:58.209060 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a808fd50-abae-49f9-b63f-517a799802c7 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:58.231336 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a808fd50-abae-49f9-b63f-517a799802c7 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:58.236613 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae65ee4-e606-42ed-b90e-bef1ece1fd7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:58.237847 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae65ee4-e606-42ed-b90e-bef1ece1fd7c admin admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:58.237847 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae65ee4-e606-42ed-b90e-bef1ece1fd7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:58.237847 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae65ee4-e606-42ed-b90e-bef1ece1fd7c admin admin] PATH_INFO: `/v3/projects/33e38693dbe5490fb39cd1981274a5f6/users/1c38342973094c43b412fff9e0f460d6/roles/ef689f96442945e8a7bc8cbfee13d909` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:58.261385 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae65ee4-e606-42ed-b90e-bef1ece1fd7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=33e38693dbe5490fb39cd1981274a5f6, user_id=1c38342973094c43b412fff9e0f460d6, role_id=ef689f96442945e8a7bc8cbfee13d909)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:58.264375 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae65ee4-e606-42ed-b90e-bef1ece1fd7c admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:58.328747 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 986/1976] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:21:58 2022] PUT /identity/v3/projects/33e38693dbe5490fb39cd1981274a5f6/users/1c38342973094c43b412fff9e0f460d6/roles/ef689f96442945e8a7bc8cbfee13d909 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 00:21:58.334207 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2e4501-491a-45f6-9544-f60c4021de96 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:58.336960 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2e4501-491a-45f6-9544-f60c4021de96 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:58.336960 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2e4501-491a-45f6-9544-f60c4021de96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:58.361904 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-6a2e4501-491a-45f6-9544-f60c4021de96 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:21:58.363254 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-6a2e4501-491a-45f6-9544-f60c4021de96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:58.437396 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 991/1977] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:21:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:21:58.447137 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae65ee4-e606-42ed-b90e-bef1ece1fd7c admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:21:58.449641 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-6ae65ee4-e606-42ed-b90e-bef1ece1fd7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:21:58.517615 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae65ee4-e606-42ed-b90e-bef1ece1fd7c admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:21:58.521983 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f857acde-29d5-49b7-ba64-8447f4327675 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:21:58.522793 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f857acde-29d5-49b7-ba64-8447f4327675 service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:21:58.523076 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f857acde-29d5-49b7-ba64-8447f4327675 service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:21:58.523425 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f857acde-29d5-49b7-ba64-8447f4327675 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:21:58.535599 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f857acde-29d5-49b7-ba64-8447f4327675 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:21:58.537313 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f857acde-29d5-49b7-ba64-8447f4327675 service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:21:58.548023 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 987/1978] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:21:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:22:04.300790 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ba410-010a-45ba-ba7c-e51b70035a37 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:22:04.301272 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ba410-010a-45ba-ba7c-e51b70035a37 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:22:04.301364 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ba410-010a-45ba-ba7c-e51b70035a37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:22:04.360648 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-353ba410-010a-45ba-ba7c-e51b70035a37 None None] MFA Rules not processed for user `1c38342973094c43b412fff9e0f460d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:22:04.362308 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-353ba410-010a-45ba-ba7c-e51b70035a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:22:04.475414 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 992/1979] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:22:04 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:22:04.477224 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f857acde-29d5-49b7-ba64-8447f4327675 service neutron] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:22:04.487432 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f857acde-29d5-49b7-ba64-8447f4327675 service neutron] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:22:04.492152 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe96f96e-5813-4d99-93e1-5e2d06f27dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:22:04.492785 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe96f96e-5813-4d99-93e1-5e2d06f27dc8 admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:22:04.493061 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe96f96e-5813-4d99-93e1-5e2d06f27dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:22:04.493349 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe96f96e-5813-4d99-93e1-5e2d06f27dc8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:22:04.494006 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe96f96e-5813-4d99-93e1-5e2d06f27dc8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:22:04.496055 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe96f96e-5813-4d99-93e1-5e2d06f27dc8 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:22:04.613356 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 988/1980] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:22:04 2022] POST /identity/v3/projects => generated 385 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:22:04.617842 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353ba410-010a-45ba-ba7c-e51b70035a37 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:22:04.631265 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353ba410-010a-45ba-ba7c-e51b70035a37 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:22:04.692459 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:22:04.714720 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13d99fb-1fdb-4560-a935-c7ea0c6ce0c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:22:04.715707 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13d99fb-1fdb-4560-a935-c7ea0c6ce0c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:22:04.716178 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13d99fb-1fdb-4560-a935-c7ea0c6ce0c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:22:04.718270 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13d99fb-1fdb-4560-a935-c7ea0c6ce0c2 admin admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:22:04.718270 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13d99fb-1fdb-4560-a935-c7ea0c6ce0c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:22:04.719267 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13d99fb-1fdb-4560-a935-c7ea0c6ce0c2 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:22:04.817777 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 993/1981] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:22:04 2022] POST /identity/v3/users => generated 299 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 00:22:04.825082 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe96f96e-5813-4d99-93e1-5e2d06f27dc8 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:22:04.843872 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe96f96e-5813-4d99-93e1-5e2d06f27dc8 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:22:04.850245 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a0403f-5072-4017-ae39-b47f7b907329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:22:04.850868 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a0403f-5072-4017-ae39-b47f7b907329 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:22:04.851177 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a0403f-5072-4017-ae39-b47f7b907329 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:22:04.851547 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a0403f-5072-4017-ae39-b47f7b907329 admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:22:04.852200 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a0403f-5072-4017-ae39-b47f7b907329 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:22:04.853815 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a0403f-5072-4017-ae39-b47f7b907329 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:22:04.864815 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 989/1982] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:22:04 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:22:04.870076 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13d99fb-1fdb-4560-a935-c7ea0c6ce0c2 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:22:04.882389 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13d99fb-1fdb-4560-a935-c7ea0c6ce0c2 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:22:04.889118 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cc2fae-71ac-4999-9daf-8eb19e9d6d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:22:04.889797 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cc2fae-71ac-4999-9daf-8eb19e9d6d9c admin admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:22:04.890076 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cc2fae-71ac-4999-9daf-8eb19e9d6d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:22:04.890351 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cc2fae-71ac-4999-9daf-8eb19e9d6d9c admin admin] PATH_INFO: `/v3/projects/a56e1912ce5649e19a687b8f0ee0e566/users/a329566772264f18a91449cf95de5709/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:22:04.909666 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cc2fae-71ac-4999-9daf-8eb19e9d6d9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a56e1912ce5649e19a687b8f0ee0e566, user_id=a329566772264f18a91449cf95de5709, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:22:04.912813 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cc2fae-71ac-4999-9daf-8eb19e9d6d9c admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:22:05.005311 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 994/1983] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:22:04 2022] PUT /identity/v3/projects/a56e1912ce5649e19a687b8f0ee0e566/users/a329566772264f18a91449cf95de5709/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:22:05.009282 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a0403f-5072-4017-ae39-b47f7b907329 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:22:05.019753 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a0403f-5072-4017-ae39-b47f7b907329 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:22:05.028367 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:22:05.078809 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74826308-b4fb-4cda-9f92-1295a3f2aeb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:22:05.079446 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74826308-b4fb-4cda-9f92-1295a3f2aeb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:22:05.079723 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74826308-b4fb-4cda-9f92-1295a3f2aeb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:22:05.080028 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74826308-b4fb-4cda-9f92-1295a3f2aeb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:22:05.080664 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74826308-b4fb-4cda-9f92-1295a3f2aeb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:22:05.082310 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74826308-b4fb-4cda-9f92-1295a3f2aeb0 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:22:05.092102 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 990/1984] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:22:05 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:22:05.096169 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cc2fae-71ac-4999-9daf-8eb19e9d6d9c admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:22:05.107112 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cc2fae-71ac-4999-9daf-8eb19e9d6d9c admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:22:05.112275 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bc33f2-77bb-40e1-ad5a-51c88c7d0f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:22:05.112920 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bc33f2-77bb-40e1-ad5a-51c88c7d0f79 admin admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:22:05.113196 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bc33f2-77bb-40e1-ad5a-51c88c7d0f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:22:05.113466 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bc33f2-77bb-40e1-ad5a-51c88c7d0f79 admin admin] PATH_INFO: `/v3/projects/a56e1912ce5649e19a687b8f0ee0e566/users/a329566772264f18a91449cf95de5709/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:22:05.115498 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bc33f2-77bb-40e1-ad5a-51c88c7d0f79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a56e1912ce5649e19a687b8f0ee0e566, user_id=a329566772264f18a91449cf95de5709, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:22:05.118166 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bc33f2-77bb-40e1-ad5a-51c88c7d0f79 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:22:05.129859 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 995/1985] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:22:05 2022] PUT /identity/v3/projects/a56e1912ce5649e19a687b8f0ee0e566/users/a329566772264f18a91449cf95de5709/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:22:09.727334 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8e57f-7ff1-46de-9cf9-8693161c2db8 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:22:09.727935 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8e57f-7ff1-46de-9cf9-8693161c2db8 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:22:09.727935 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8e57f-7ff1-46de-9cf9-8693161c2db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:22:09.752740 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-31a8e57f-7ff1-46de-9cf9-8693161c2db8 None None] MFA Rules not processed for user `cdd87151f3af4a3bb1c3815db5d0e8fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:22:09.753976 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-31a8e57f-7ff1-46de-9cf9-8693161c2db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:22:09.815160 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 991/1986] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:22:09 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:22:13.321339 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1314771d-c740-40dc-a296-ab5b0bcc631d None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:22:13.322449 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1314771d-c740-40dc-a296-ab5b0bcc631d None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:22:13.322449 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1314771d-c740-40dc-a296-ab5b0bcc631d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:22:13.364646 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-1314771d-c740-40dc-a296-ab5b0bcc631d None None] MFA Rules not processed for user `a329566772264f18a91449cf95de5709`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:22:13.365911 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-1314771d-c740-40dc-a296-ab5b0bcc631d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:22:13.486470 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 996/1987] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:22:13 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:22:13.497297 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8e57f-7ff1-46de-9cf9-8693161c2db8 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:22:13.498827 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-31a8e57f-7ff1-46de-9cf9-8693161c2db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:22:13.600461 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8e57f-7ff1-46de-9cf9-8693161c2db8 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:22:13.605653 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270334fe-2fee-4c92-a2a3-26021883d9c8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59b883abbb674b3c976bacb64194a66d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:22:13.606286 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270334fe-2fee-4c92-a2a3-26021883d9c8 service glance] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:22:13.607599 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270334fe-2fee-4c92-a2a3-26021883d9c8 service glance] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:22:13.607599 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270334fe-2fee-4c92-a2a3-26021883d9c8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:22:13.671199 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:22:13.688104 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270334fe-2fee-4c92-a2a3-26021883d9c8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:22:13.689578 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270334fe-2fee-4c92-a2a3-26021883d9c8 service glance] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:22:13.708012 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 992/1988] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:22:13 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:22:14.130080 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cc38f5-e2eb-4ac0-9651-8298bd3e753f None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:22:14.130365 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cc38f5-e2eb-4ac0-9651-8298bd3e753f None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:22:14.130639 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cc38f5-e2eb-4ac0-9651-8298bd3e753f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:22:14.151004 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-15cc38f5-e2eb-4ac0-9651-8298bd3e753f None None] MFA Rules not processed for user `d030d18b595144f19d6ad01d4cd5d52b`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:22:14.152280 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-15cc38f5-e2eb-4ac0-9651-8298bd3e753f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:22:14.224993 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 997/1989] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Oct 12 00:22:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:22:14.273867 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270334fe-2fee-4c92-a2a3-26021883d9c8 service glance] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:22:14.276006 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-270334fe-2fee-4c92-a2a3-26021883d9c8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:22:14.377023 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270334fe-2fee-4c92-a2a3-26021883d9c8 service glance] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:22:14.391246 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71600185-19bd-42ef-b8a8-77e73adf5245 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc1bcffc78844dcbb1a9b7c80285249b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:22:14.391924 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71600185-19bd-42ef-b8a8-77e73adf5245 service swift] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:22:14.392210 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71600185-19bd-42ef-b8a8-77e73adf5245 service swift] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:22:14.392484 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71600185-19bd-42ef-b8a8-77e73adf5245 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:22:14.417305 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71600185-19bd-42ef-b8a8-77e73adf5245 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:22:14.422052 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71600185-19bd-42ef-b8a8-77e73adf5245 service swift] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:22:14.435966 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 993/1990] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Oct 12 00:22:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 12 00:22:24.216431 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=84,keystone:INSERT=5 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:22:24.430468 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=38 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:22:27.090922 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c8d439-f672-4c7a-b652-2c08d87807cf None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:22:27.091488 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c8d439-f672-4c7a-b652-2c08d87807cf None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:22:27.091488 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c8d439-f672-4c7a-b652-2c08d87807cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:22:27.110861 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-18c8d439-f672-4c7a-b652-2c08d87807cf None None] MFA Rules not processed for user `1c38342973094c43b412fff9e0f460d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:22:27.112091 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-18c8d439-f672-4c7a-b652-2c08d87807cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:22:27.119021 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 998/1991] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:22:27 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:22:35.797956 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195d7281-bfca-4933-bd9c-9e913f000bcf None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:22:35.798543 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195d7281-bfca-4933-bd9c-9e913f000bcf None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:22:35.798543 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195d7281-bfca-4933-bd9c-9e913f000bcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:22:35.819098 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-195d7281-bfca-4933-bd9c-9e913f000bcf None None] MFA Rules not processed for user `d030d18b595144f19d6ad01d4cd5d52b`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:22:35.820563 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-195d7281-bfca-4933-bd9c-9e913f000bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:22:35.829390 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 994/1992] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Oct 12 00:22:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:22:35.842473 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c8d439-f672-4c7a-b652-2c08d87807cf None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:22:35.854960 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c8d439-f672-4c7a-b652-2c08d87807cf None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:22:35.865858 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc50ae9-6d6f-4ee4-be71-896c15cbdeed service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc1bcffc78844dcbb1a9b7c80285249b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:22:35.868803 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc50ae9-6d6f-4ee4-be71-896c15cbdeed service swift] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:22:35.868803 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc50ae9-6d6f-4ee4-be71-896c15cbdeed service swift] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:22:35.868803 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc50ae9-6d6f-4ee4-be71-896c15cbdeed service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:22:35.876764 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc50ae9-6d6f-4ee4-be71-896c15cbdeed service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:22:35.878596 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc50ae9-6d6f-4ee4-be71-896c15cbdeed service swift] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:22:35.909240 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 999/1993] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Oct 12 00:22:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 12 00:22:45.813406 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=3 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:22:45.884011 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=9 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:23:07.641220 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cdbcab-9cf2-4795-b10a-89829c1b021d None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:23:07.641692 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cdbcab-9cf2-4795-b10a-89829c1b021d None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:23:07.641801 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cdbcab-9cf2-4795-b10a-89829c1b021d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:23:07.661112 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-48cdbcab-9cf2-4795-b10a-89829c1b021d None None] MFA Rules not processed for user `b0791b4507cf48bca517f41f0feebd78`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:23:07.662438 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-48cdbcab-9cf2-4795-b10a-89829c1b021d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:23:07.717850 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 995/1994] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:23:07 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:23:10.431253 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc50ae9-6d6f-4ee4-be71-896c15cbdeed service swift] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:23:10.432685 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-bdc50ae9-6d6f-4ee4-be71-896c15cbdeed service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:23:10.500834 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc50ae9-6d6f-4ee4-be71-896c15cbdeed service swift] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:23:10.506217 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cae83b7-d10c-46c0-8aae-c698cc2292ac service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e528da752ed4c78bc747215909f6569', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:23:10.506832 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cae83b7-d10c-46c0-8aae-c698cc2292ac service placement] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:23:10.507116 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cae83b7-d10c-46c0-8aae-c698cc2292ac service placement] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:23:10.507379 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cae83b7-d10c-46c0-8aae-c698cc2292ac service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:23:10.509871 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-1cae83b7-d10c-46c0-8aae-c698cc2292ac service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:23:10.521648 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cae83b7-d10c-46c0-8aae-c698cc2292ac service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:23:10.523224 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cae83b7-d10c-46c0-8aae-c698cc2292ac service placement] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:23:10.533170 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1000/1995] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:23:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:23:15.340076 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cdbcab-9cf2-4795-b10a-89829c1b021d None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:23:15.341482 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-48cdbcab-9cf2-4795-b10a-89829c1b021d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:23:15.397740 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cdbcab-9cf2-4795-b10a-89829c1b021d None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:23:15.401362 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bdd688-0ae1-42ec-b8d4-94602aecc4c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:23:15.401961 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bdd688-0ae1-42ec-b8d4-94602aecc4c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:23:15.402260 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bdd688-0ae1-42ec-b8d4-94602aecc4c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:23:15.402527 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bdd688-0ae1-42ec-b8d4-94602aecc4c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:23:15.411702 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bdd688-0ae1-42ec-b8d4-94602aecc4c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:23:15.413196 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bdd688-0ae1-42ec-b8d4-94602aecc4c2 service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:23:15.422998 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 996/1996] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:23:15 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:23:20.528892 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=32 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:23:22.806455 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e3423-d0fc-4642-9b62-198e343bd8c3 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:23:22.807014 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e3423-d0fc-4642-9b62-198e343bd8c3 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:23:22.807014 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e3423-d0fc-4642-9b62-198e343bd8c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:23:22.826517 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-966e3423-d0fc-4642-9b62-198e343bd8c3 None None] MFA Rules not processed for user `d030d18b595144f19d6ad01d4cd5d52b`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:23:22.827762 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-966e3423-d0fc-4642-9b62-198e343bd8c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:23:22.834785 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1001/1997] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Oct 12 00:23:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 00:23:22.860481 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bdd688-0ae1-42ec-b8d4-94602aecc4c2 service neutron] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:23:22.884494 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bdd688-0ae1-42ec-b8d4-94602aecc4c2 service neutron] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:23:22.890506 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcc7aff-4385-42c0-9e5a-0aca3e27b496 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc1bcffc78844dcbb1a9b7c80285249b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:23:22.891119 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc7aff-4385-42c0-9e5a-0aca3e27b496 service swift] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:23:22.891429 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc7aff-4385-42c0-9e5a-0aca3e27b496 service swift] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:23:22.891703 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc7aff-4385-42c0-9e5a-0aca3e27b496 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:23:22.900998 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcc7aff-4385-42c0-9e5a-0aca3e27b496 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:23:22.902605 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcc7aff-4385-42c0-9e5a-0aca3e27b496 service swift] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:23:22.910704 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 997/1998] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Oct 12 00:23:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 12 00:23:32.821214 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=3 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:23:32.908916 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=57 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:23:45.967984 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2972d5d8-2031-474c-b9ad-59cb8f9a7fe7 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:23:45.968408 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2972d5d8-2031-474c-b9ad-59cb8f9a7fe7 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:23:45.968756 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2972d5d8-2031-474c-b9ad-59cb8f9a7fe7 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:23:45.973400 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1002/1999] 10.222.0.36 () {54 vars in 921 bytes} [Wed Oct 12 00:23:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:23:45.979797 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07fd9c7-0c5e-4113-8a09-71df9cf43d6d None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:23:45.981310 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07fd9c7-0c5e-4113-8a09-71df9cf43d6d None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:23:45.981595 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07fd9c7-0c5e-4113-8a09-71df9cf43d6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:23:46.005557 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-e07fd9c7-0c5e-4113-8a09-71df9cf43d6d None None] MFA Rules not processed for user `a515b75aa3944912ad2364ffd7924f40`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:23:46.006952 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-e07fd9c7-0c5e-4113-8a09-71df9cf43d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:23:46.082536 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 998/2000] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Oct 12 00:23:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:23:46.094189 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2972d5d8-2031-474c-b9ad-59cb8f9a7fe7 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:23:46.112435 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2972d5d8-2031-474c-b9ad-59cb8f9a7fe7 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:23:46.116366 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e0d406-e365-4120-a6f7-d19ff7970ade service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59b883abbb674b3c976bacb64194a66d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:23:46.117094 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e0d406-e365-4120-a6f7-d19ff7970ade service glance] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:23:46.117461 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e0d406-e365-4120-a6f7-d19ff7970ade service glance] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:23:46.117798 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e0d406-e365-4120-a6f7-d19ff7970ade service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:23:46.129239 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e0d406-e365-4120-a6f7-d19ff7970ade service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:23:46.130685 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e0d406-e365-4120-a6f7-d19ff7970ade service glance] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:23:46.143584 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1003/2001] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:23:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:23:46.348373 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa92dae-406f-4d03-8a11-6b39074eea21 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:23:46.348653 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa92dae-406f-4d03-8a11-6b39074eea21 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:23:46.348922 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa92dae-406f-4d03-8a11-6b39074eea21 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:23:46.350371 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 999/2002] 10.222.0.36 () {54 vars in 921 bytes} [Wed Oct 12 00:23:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:23:46.355354 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e386a35f-1931-47bd-837e-0d55b6e931cc None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:23:46.355679 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e386a35f-1931-47bd-837e-0d55b6e931cc None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:23:46.355979 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e386a35f-1931-47bd-837e-0d55b6e931cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:23:46.378355 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-e386a35f-1931-47bd-837e-0d55b6e931cc None None] MFA Rules not processed for user `51215dab618141b4991e232724094140`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:23:46.379630 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-e386a35f-1931-47bd-837e-0d55b6e931cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:23:46.386453 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1004/2003] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Oct 12 00:23:46 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:23:46.400188 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa92dae-406f-4d03-8a11-6b39074eea21 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:23:46.411159 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa92dae-406f-4d03-8a11-6b39074eea21 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:23:46.415460 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6014580-76a9-4f38-ab4e-06bfae09a647 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:23:46.416090 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6014580-76a9-4f38-ab4e-06bfae09a647 service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:23:46.416368 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6014580-76a9-4f38-ab4e-06bfae09a647 service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:23:46.416640 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6014580-76a9-4f38-ab4e-06bfae09a647 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:23:46.423840 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6014580-76a9-4f38-ab4e-06bfae09a647 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:23:46.425341 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6014580-76a9-4f38-ab4e-06bfae09a647 service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:23:46.433038 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1000/2004] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:23:46 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:23:46.517711 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57d6af4-0a83-4cec-8fb9-5a8b1735ef86 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:23:46.517990 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57d6af4-0a83-4cec-8fb9-5a8b1735ef86 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:23:46.518257 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57d6af4-0a83-4cec-8fb9-5a8b1735ef86 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:23:46.519656 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1005/2005] 10.222.0.36 () {54 vars in 921 bytes} [Wed Oct 12 00:23:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:23:46.524528 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ffd7c5-e036-4f33-b85c-3af659acecb4 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:23:46.524813 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ffd7c5-e036-4f33-b85c-3af659acecb4 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:23:46.525088 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ffd7c5-e036-4f33-b85c-3af659acecb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:23:46.545186 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-52ffd7c5-e036-4f33-b85c-3af659acecb4 None None] MFA Rules not processed for user `a515b75aa3944912ad2364ffd7924f40`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:23:46.546403 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-52ffd7c5-e036-4f33-b85c-3af659acecb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:23:46.556726 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1001/2006] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Oct 12 00:23:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:23:46.567869 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57d6af4-0a83-4cec-8fb9-5a8b1735ef86 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:23:46.580220 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57d6af4-0a83-4cec-8fb9-5a8b1735ef86 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:23:46.584386 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829e5c15-f274-454b-8ee0-e093c94f0e9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:23:46.584990 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829e5c15-f274-454b-8ee0-e093c94f0e9a service neutron] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:23:46.585259 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829e5c15-f274-454b-8ee0-e093c94f0e9a service neutron] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:23:46.585691 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829e5c15-f274-454b-8ee0-e093c94f0e9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:23:46.596281 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829e5c15-f274-454b-8ee0-e093c94f0e9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:23:46.597709 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829e5c15-f274-454b-8ee0-e093c94f0e9a service neutron] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:23:46.608795 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1006/2007] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:23:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:23:56.540046 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=40 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:23:56.604009 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=15 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:23:57.432528 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600f7a19-8ef9-4dd5-be92-c43565d7e2b6 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:23:57.432799 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600f7a19-8ef9-4dd5-be92-c43565d7e2b6 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:23:57.433612 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600f7a19-8ef9-4dd5-be92-c43565d7e2b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:23:57.454031 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-600f7a19-8ef9-4dd5-be92-c43565d7e2b6 None None] MFA Rules not processed for user `d030d18b595144f19d6ad01d4cd5d52b`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:23:57.455269 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-600f7a19-8ef9-4dd5-be92-c43565d7e2b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:23:57.462910 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1002/2008] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Oct 12 00:23:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:23:57.476773 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829e5c15-f274-454b-8ee0-e093c94f0e9a service neutron] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:23:57.490011 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829e5c15-f274-454b-8ee0-e093c94f0e9a service neutron] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:23:57.496691 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3251a7-93e0-4cce-b42c-dd130545c9b2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc1bcffc78844dcbb1a9b7c80285249b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:23:57.497390 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3251a7-93e0-4cce-b42c-dd130545c9b2 service swift] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:23:57.497732 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3251a7-93e0-4cce-b42c-dd130545c9b2 service swift] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:23:57.498069 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3251a7-93e0-4cce-b42c-dd130545c9b2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:23:57.507302 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3251a7-93e0-4cce-b42c-dd130545c9b2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:23:57.510032 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3251a7-93e0-4cce-b42c-dd130545c9b2 service swift] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:23:57.518547 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1007/2009] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Oct 12 00:23:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 12 00:24:00.593981 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e4b269-3bb2-4414-abe6-9be23e6b4d3c None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:24:00.594567 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e4b269-3bb2-4414-abe6-9be23e6b4d3c None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:24:00.594567 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e4b269-3bb2-4414-abe6-9be23e6b4d3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:24:00.614454 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-80e4b269-3bb2-4414-abe6-9be23e6b4d3c None None] MFA Rules not processed for user `de766cb582ba4c18b1fda9bf4ad43f65`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:24:00.616045 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-80e4b269-3bb2-4414-abe6-9be23e6b4d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:24:00.673092 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1003/2010] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:24:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:24:07.514797 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=6 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:24:10.666196 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=28 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:24:15.337576 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8c1dba-2920-4890-810f-2c2db61f6c77 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:24:15.338800 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8c1dba-2920-4890-810f-2c2db61f6c77 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:24:15.338800 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8c1dba-2920-4890-810f-2c2db61f6c77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:24:15.358534 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-cb8c1dba-2920-4890-810f-2c2db61f6c77 None None] MFA Rules not processed for user `cdd87151f3af4a3bb1c3815db5d0e8fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:24:15.359787 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-cb8c1dba-2920-4890-810f-2c2db61f6c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:24:15.367091 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1008/2011] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:24:15 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:24:17.913811 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c973cb25-f13a-41e1-b7c1-282c1d992b1b None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:24:17.914674 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c973cb25-f13a-41e1-b7c1-282c1d992b1b None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:24:17.915148 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c973cb25-f13a-41e1-b7c1-282c1d992b1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:24:17.937154 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-c973cb25-f13a-41e1-b7c1-282c1d992b1b None None] MFA Rules not processed for user `d030d18b595144f19d6ad01d4cd5d52b`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:24:17.938412 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-c973cb25-f13a-41e1-b7c1-282c1d992b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:24:17.946233 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1004/2012] 10.222.0.36 () {58 vars in 967 bytes} [Wed Oct 12 00:24:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:24:17.959145 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8c1dba-2920-4890-810f-2c2db61f6c77 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:24:17.973124 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8c1dba-2920-4890-810f-2c2db61f6c77 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:24:17.979810 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12660b2-9890-45fb-be9e-f72f8f92241f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc1bcffc78844dcbb1a9b7c80285249b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:24:17.980724 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12660b2-9890-45fb-be9e-f72f8f92241f service swift] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:24:17.984170 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12660b2-9890-45fb-be9e-f72f8f92241f service swift] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:24:17.984170 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12660b2-9890-45fb-be9e-f72f8f92241f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:24:17.990749 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12660b2-9890-45fb-be9e-f72f8f92241f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:24:17.992372 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12660b2-9890-45fb-be9e-f72f8f92241f service swift] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:24:18.007721 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1009/2013] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Oct 12 00:24:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 12 00:24:25.523512 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c973cb25-f13a-41e1-b7c1-282c1d992b1b None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:24:25.524942 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-c973cb25-f13a-41e1-b7c1-282c1d992b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:24:25.598885 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c973cb25-f13a-41e1-b7c1-282c1d992b1b None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:24:25.603548 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2c8e99-2dc8-4441-8ff7-0ec2c24fa32e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:24:25.604187 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c8e99-2dc8-4441-8ff7-0ec2c24fa32e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:24:25.604468 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c8e99-2dc8-4441-8ff7-0ec2c24fa32e admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:24:25.604741 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c8e99-2dc8-4441-8ff7-0ec2c24fa32e admin admin] PATH_INFO: `/v3/users/b0791b4507cf48bca517f41f0feebd78` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:24:25.605929 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2c8e99-2dc8-4441-8ff7-0ec2c24fa32e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0791b4507cf48bca517f41f0feebd78)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:24:25.607861 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2c8e99-2dc8-4441-8ff7-0ec2c24fa32e admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:24:25.824522 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-cd2c8e99-2dc8-4441-8ff7-0ec2c24fa32e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0791b4507cf48bca517f41f0feebd78'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:24:25.825275 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-cd2c8e99-2dc8-4441-8ff7-0ec2c24fa32e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0791b4507cf48bca517f41f0feebd78'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:24:25.837804 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-cd2c8e99-2dc8-4441-8ff7-0ec2c24fa32e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0791b4507cf48bca517f41f0feebd78'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:24:25.899806 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-cd2c8e99-2dc8-4441-8ff7-0ec2c24fa32e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0791b4507cf48bca517f41f0feebd78'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:24:25.900551 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-cd2c8e99-2dc8-4441-8ff7-0ec2c24fa32e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0791b4507cf48bca517f41f0feebd78'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:24:25.914373 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1005/2014] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:24:25 2022] DELETE /identity/v3/users/b0791b4507cf48bca517f41f0feebd78 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:24:26.243875 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12660b2-9890-45fb-be9e-f72f8f92241f service swift] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:24:26.254564 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12660b2-9890-45fb-be9e-f72f8f92241f service swift] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:24:26.323692 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17226dcb-8225-4d59-b64e-e19021e1ef89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:24:26.324438 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17226dcb-8225-4d59-b64e-e19021e1ef89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:24:26.324785 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17226dcb-8225-4d59-b64e-e19021e1ef89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:24:26.325120 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17226dcb-8225-4d59-b64e-e19021e1ef89 admin admin] PATH_INFO: `/v3/projects/4194baa4b1fe4ab2840720dbf012a2f3` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:24:26.326406 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17226dcb-8225-4d59-b64e-e19021e1ef89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4194baa4b1fe4ab2840720dbf012a2f3)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:24:26.328529 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17226dcb-8225-4d59-b64e-e19021e1ef89 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:24:26.467689 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-17226dcb-8225-4d59-b64e-e19021e1ef89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4194baa4b1fe4ab2840720dbf012a2f3', 'request_id': 'req-17226dcb-8225-4d59-b64e-e19021e1ef89'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:24:26.473884 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-17226dcb-8225-4d59-b64e-e19021e1ef89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4194baa4b1fe4ab2840720dbf012a2f3', 'request_id': 'req-17226dcb-8225-4d59-b64e-e19021e1ef89'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:24:26.551493 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-17226dcb-8225-4d59-b64e-e19021e1ef89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4194baa4b1fe4ab2840720dbf012a2f3', 'request_id': 'req-17226dcb-8225-4d59-b64e-e19021e1ef89'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:24:26.562628 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-17226dcb-8225-4d59-b64e-e19021e1ef89 admin admin] The token cache is being invalidate because project 4194baa4b1fe4ab2840720dbf012a2f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81747) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:24:26.562918 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-17226dcb-8225-4d59-b64e-e19021e1ef89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:24:26.563566 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-17226dcb-8225-4d59-b64e-e19021e1ef89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:24:26.565128 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1010/2015] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:24:26 2022] DELETE /identity/v3/projects/4194baa4b1fe4ab2840720dbf012a2f3 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:24:35.910611 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=56,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:24:36.559536 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=60,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:24:59.709782 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2c8e99-2dc8-4441-8ff7-0ec2c24fa32e admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:24:59.711168 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-cd2c8e99-2dc8-4441-8ff7-0ec2c24fa32e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:24:59.767876 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2c8e99-2dc8-4441-8ff7-0ec2c24fa32e admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:24:59.815163 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34412f18-ad39-4159-96ec-8086d6e2c5d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:24:59.815795 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34412f18-ad39-4159-96ec-8086d6e2c5d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:24:59.816118 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34412f18-ad39-4159-96ec-8086d6e2c5d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:24:59.816379 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34412f18-ad39-4159-96ec-8086d6e2c5d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:24:59.818067 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-34412f18-ad39-4159-96ec-8086d6e2c5d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:24:59.887379 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34412f18-ad39-4159-96ec-8086d6e2c5d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:24:59.889100 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34412f18-ad39-4159-96ec-8086d6e2c5d4 service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:24:59.899914 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1006/2016] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:24:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:25:01.254172 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17226dcb-8225-4d59-b64e-e19021e1ef89 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:25:01.255603 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-17226dcb-8225-4d59-b64e-e19021e1ef89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:25:01.316272 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17226dcb-8225-4d59-b64e-e19021e1ef89 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:25:01.319858 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ed25c2-78bf-40f0-90d2-ced62e8fe777 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:25:01.320610 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ed25c2-78bf-40f0-90d2-ced62e8fe777 service cinder] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:25:01.320890 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ed25c2-78bf-40f0-90d2-ced62e8fe777 service cinder] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:25:01.321161 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ed25c2-78bf-40f0-90d2-ced62e8fe777 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:25:01.323031 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-80ed25c2-78bf-40f0-90d2-ced62e8fe777 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:25:01.337047 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ed25c2-78bf-40f0-90d2-ced62e8fe777 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:25:01.338512 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ed25c2-78bf-40f0-90d2-ced62e8fe777 service cinder] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:25:01.352043 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1011/2017] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:25:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:25:07.914564 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34412f18-ad39-4159-96ec-8086d6e2c5d4 service neutron] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:25:07.916337 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-34412f18-ad39-4159-96ec-8086d6e2c5d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:25:07.931823 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34412f18-ad39-4159-96ec-8086d6e2c5d4 service neutron] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:25:07.937921 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf89ec4-244d-4dd1-b728-a35751b552f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a515b75aa3944912ad2364ffd7924f40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:25:07.938530 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf89ec4-244d-4dd1-b728-a35751b552f6 service nova] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:25:07.938801 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf89ec4-244d-4dd1-b728-a35751b552f6 service nova] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:25:07.939067 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf89ec4-244d-4dd1-b728-a35751b552f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:25:07.940826 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-fcf89ec4-244d-4dd1-b728-a35751b552f6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:25:08.020844 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:25:08.023306 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf89ec4-244d-4dd1-b728-a35751b552f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:25:08.025019 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf89ec4-244d-4dd1-b728-a35751b552f6 service nova] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:25:08.036248 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1007/2018] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:25:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:25:08.400198 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ed25c2-78bf-40f0-90d2-ced62e8fe777 service cinder] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:25:08.402479 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-80ed25c2-78bf-40f0-90d2-ced62e8fe777 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:25:08.418426 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ed25c2-78bf-40f0-90d2-ced62e8fe777 service cinder] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:25:08.424888 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84117daa-f02c-4688-8877-e7f6e3c961b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a515b75aa3944912ad2364ffd7924f40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:25:08.425634 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84117daa-f02c-4688-8877-e7f6e3c961b9 service nova] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:25:08.426012 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84117daa-f02c-4688-8877-e7f6e3c961b9 service nova] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:25:08.426384 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84117daa-f02c-4688-8877-e7f6e3c961b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:25:08.428302 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-84117daa-f02c-4688-8877-e7f6e3c961b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:25:08.544461 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84117daa-f02c-4688-8877-e7f6e3c961b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:25:08.546171 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84117daa-f02c-4688-8877-e7f6e3c961b9 service nova] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:25:08.555938 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1012/2019] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:25:08 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:25:11.835052 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1777b74e-8123-4450-84d0-a43deee82707 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:25:11.835625 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1777b74e-8123-4450-84d0-a43deee82707 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:25:11.835838 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1777b74e-8123-4450-84d0-a43deee82707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:25:11.857237 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-1777b74e-8123-4450-84d0-a43deee82707 None None] MFA Rules not processed for user `bca53aca7b134df1949deb10858f14d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:25:11.858831 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-1777b74e-8123-4450-84d0-a43deee82707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:25:11.866887 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1008/2020] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:25:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:25:12.361123 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84117daa-f02c-4688-8877-e7f6e3c961b9 service nova] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:25:12.362645 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-84117daa-f02c-4688-8877-e7f6e3c961b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:25:12.374922 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84117daa-f02c-4688-8877-e7f6e3c961b9 service nova] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:25:12.378700 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66da60e4-4341-4716-be5f-8e75636c03b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:25:12.379297 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66da60e4-4341-4716-be5f-8e75636c03b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:25:12.379521 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66da60e4-4341-4716-be5f-8e75636c03b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:25:12.379790 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66da60e4-4341-4716-be5f-8e75636c03b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:25:12.381668 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-66da60e4-4341-4716-be5f-8e75636c03b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:25:12.392933 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66da60e4-4341-4716-be5f-8e75636c03b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:25:12.394652 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66da60e4-4341-4716-be5f-8e75636c03b0 service neutron] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:25:12.403350 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1013/2021] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:25:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:25:17.063934 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1777b74e-8123-4450-84d0-a43deee82707 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:25:17.079411 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1777b74e-8123-4450-84d0-a43deee82707 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:25:17.085577 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffee847f-0bf4-41e3-80c7-64f744fde4ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a515b75aa3944912ad2364ffd7924f40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:25:17.086325 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffee847f-0bf4-41e3-80c7-64f744fde4ad service nova] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:25:17.086677 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffee847f-0bf4-41e3-80c7-64f744fde4ad service nova] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:25:17.087020 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffee847f-0bf4-41e3-80c7-64f744fde4ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:25:17.089112 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-ffee847f-0bf4-41e3-80c7-64f744fde4ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:25:17.160013 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffee847f-0bf4-41e3-80c7-64f744fde4ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:25:17.162268 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffee847f-0bf4-41e3-80c7-64f744fde4ad service nova] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:25:17.174860 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1009/2022] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:25:17 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:25:18.844935 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7597d61-685b-41b1-b9cf-67c18a52861a None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:25:18.845884 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7597d61-685b-41b1-b9cf-67c18a52861a None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:25:18.846358 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7597d61-685b-41b1-b9cf-67c18a52861a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:25:18.869263 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-a7597d61-685b-41b1-b9cf-67c18a52861a None None] MFA Rules not processed for user `63474c55edbf4b22ad609471a2414ede`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:25:18.870743 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-a7597d61-685b-41b1-b9cf-67c18a52861a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:25:18.962578 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1014/2023] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:25:18 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:25:20.168323 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f27e532-f6b5-452c-9399-9c8e88c5fb73 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:25:20.169773 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f27e532-f6b5-452c-9399-9c8e88c5fb73 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:25:20.169773 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f27e532-f6b5-452c-9399-9c8e88c5fb73 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:25:20.172144 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1010/2024] 10.222.0.36 () {54 vars in 961 bytes} [Wed Oct 12 00:25:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:25:20.181574 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca418d1-a887-4c33-b1c5-5f237dd67d99 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:25:20.182088 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca418d1-a887-4c33-b1c5-5f237dd67d99 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:25:20.182167 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca418d1-a887-4c33-b1c5-5f237dd67d99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:25:20.214403 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-7ca418d1-a887-4c33-b1c5-5f237dd67d99 None None] MFA Rules not processed for user `1e528da752ed4c78bc747215909f6569`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:25:20.215857 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-7ca418d1-a887-4c33-b1c5-5f237dd67d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:25:20.256351 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:25:20.311817 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1015/2025] 10.222.0.36 () {58 vars in 1058 bytes} [Wed Oct 12 00:25:20 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:25:20.320686 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f27e532-f6b5-452c-9399-9c8e88c5fb73 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:25:20.344254 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f27e532-f6b5-452c-9399-9c8e88c5fb73 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:25:20.349420 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee18531-947e-4de2-95ce-379f88cb3107 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e528da752ed4c78bc747215909f6569', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:25:20.350008 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee18531-947e-4de2-95ce-379f88cb3107 service placement] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:25:20.350227 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee18531-947e-4de2-95ce-379f88cb3107 service placement] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:25:20.350599 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee18531-947e-4de2-95ce-379f88cb3107 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:25:20.352369 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-5ee18531-947e-4de2-95ce-379f88cb3107 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:25:20.364450 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee18531-947e-4de2-95ce-379f88cb3107 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:25:20.366140 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee18531-947e-4de2-95ce-379f88cb3107 service placement] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:25:20.376045 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1011/2026] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:25:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:25:23.805968 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca418d1-a887-4c33-b1c5-5f237dd67d99 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:25:23.808293 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-7ca418d1-a887-4c33-b1c5-5f237dd67d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:25:23.824529 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca418d1-a887-4c33-b1c5-5f237dd67d99 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:25:23.832774 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a288a4c3-d79e-41cc-9dd7-2554ed4c481a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:25:23.833411 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a288a4c3-d79e-41cc-9dd7-2554ed4c481a service neutron] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:25:23.833734 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a288a4c3-d79e-41cc-9dd7-2554ed4c481a service neutron] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:25:23.834011 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a288a4c3-d79e-41cc-9dd7-2554ed4c481a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:25:23.835954 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-a288a4c3-d79e-41cc-9dd7-2554ed4c481a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:25:23.847283 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a288a4c3-d79e-41cc-9dd7-2554ed4c481a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:25:23.849175 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a288a4c3-d79e-41cc-9dd7-2554ed4c481a service neutron] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:25:23.859282 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1016/2027] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:25:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:25:25.596290 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee45ca0-ed06-4503-8f2a-419ce0a3dbfe None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:25:25.596741 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee45ca0-ed06-4503-8f2a-419ce0a3dbfe None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:25:25.596993 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee45ca0-ed06-4503-8f2a-419ce0a3dbfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:25:25.617304 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-bee45ca0-ed06-4503-8f2a-419ce0a3dbfe None None] MFA Rules not processed for user `d030d18b595144f19d6ad01d4cd5d52b`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:25:25.618601 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-bee45ca0-ed06-4503-8f2a-419ce0a3dbfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:25:25.687096 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1012/2028] 10.222.0.36 () {58 vars in 967 bytes} [Wed Oct 12 00:25:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:25:25.885305 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a288a4c3-d79e-41cc-9dd7-2554ed4c481a service neutron] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:25:25.887079 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-a288a4c3-d79e-41cc-9dd7-2554ed4c481a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:25:25.993237 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a288a4c3-d79e-41cc-9dd7-2554ed4c481a service neutron] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:25:25.998743 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c79bb7-383d-45d1-b371-bd96626d4685 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc1bcffc78844dcbb1a9b7c80285249b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:25:25.999419 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c79bb7-383d-45d1-b371-bd96626d4685 service swift] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:25:25.999698 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c79bb7-383d-45d1-b371-bd96626d4685 service swift] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:25:26.000595 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c79bb7-383d-45d1-b371-bd96626d4685 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:25:26.008907 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c79bb7-383d-45d1-b371-bd96626d4685 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:25:26.011313 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c79bb7-383d-45d1-b371-bd96626d4685 service swift] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:25:26.019113 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1017/2029] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Oct 12 00:25:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 12 00:25:28.781486 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee45ca0-ed06-4503-8f2a-419ce0a3dbfe None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:25:28.794772 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee45ca0-ed06-4503-8f2a-419ce0a3dbfe None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:25:28.801124 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df1494e-02f3-43c6-b0e7-7dfe320e2686 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a515b75aa3944912ad2364ffd7924f40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:25:28.801868 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df1494e-02f3-43c6-b0e7-7dfe320e2686 service nova] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:25:28.802206 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df1494e-02f3-43c6-b0e7-7dfe320e2686 service nova] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:25:28.802548 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df1494e-02f3-43c6-b0e7-7dfe320e2686 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:25:28.804403 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-8df1494e-02f3-43c6-b0e7-7dfe320e2686 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:25:28.817563 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df1494e-02f3-43c6-b0e7-7dfe320e2686 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:25:28.820156 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df1494e-02f3-43c6-b0e7-7dfe320e2686 service nova] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:25:28.830358 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1013/2030] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:25:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:25:33.628976 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c79bb7-383d-45d1-b371-bd96626d4685 service swift] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:25:33.642546 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c79bb7-383d-45d1-b371-bd96626d4685 service swift] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:25:33.651165 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21a51fb-f650-4126-b02d-c87ad9842719 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a515b75aa3944912ad2364ffd7924f40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:25:33.651735 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a51fb-f650-4126-b02d-c87ad9842719 service nova] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:25:33.652213 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a51fb-f650-4126-b02d-c87ad9842719 service nova] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:25:33.652642 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a51fb-f650-4126-b02d-c87ad9842719 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:25:33.655274 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-c21a51fb-f650-4126-b02d-c87ad9842719 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:25:33.673376 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21a51fb-f650-4126-b02d-c87ad9842719 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:25:33.676105 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21a51fb-f650-4126-b02d-c87ad9842719 service nova] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:25:33.692623 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1018/2031] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:25:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:25:36.384040 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df1494e-02f3-43c6-b0e7-7dfe320e2686 service nova] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:25:36.393654 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df1494e-02f3-43c6-b0e7-7dfe320e2686 service nova] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:25:36.397706 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658d679e-02de-4d2a-be64-92ea90ce51b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:25:36.398318 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658d679e-02de-4d2a-be64-92ea90ce51b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:25:36.398598 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658d679e-02de-4d2a-be64-92ea90ce51b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:25:36.398868 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658d679e-02de-4d2a-be64-92ea90ce51b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:25:36.400636 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-658d679e-02de-4d2a-be64-92ea90ce51b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:25:36.463143 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658d679e-02de-4d2a-be64-92ea90ce51b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:25:36.464636 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658d679e-02de-4d2a-be64-92ea90ce51b6 service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:25:36.470153 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:25:36.484744 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1014/2032] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:25:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 00:25:38.028681 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21a51fb-f650-4126-b02d-c87ad9842719 service nova] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:25:38.032966 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-c21a51fb-f650-4126-b02d-c87ad9842719 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:25:38.051043 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21a51fb-f650-4126-b02d-c87ad9842719 service nova] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:25:38.056457 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93763633-043f-4fb6-855b-5d646e6cef9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:25:38.057355 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93763633-043f-4fb6-855b-5d646e6cef9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:25:38.057840 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93763633-043f-4fb6-855b-5d646e6cef9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:25:38.058320 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93763633-043f-4fb6-855b-5d646e6cef9f admin admin] PATH_INFO: `/v3/users/1c38342973094c43b412fff9e0f460d6` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:25:38.059800 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93763633-043f-4fb6-855b-5d646e6cef9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c38342973094c43b412fff9e0f460d6)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:25:38.062205 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93763633-043f-4fb6-855b-5d646e6cef9f admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:25:38.277335 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-93763633-043f-4fb6-855b-5d646e6cef9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c38342973094c43b412fff9e0f460d6'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:25:38.278397 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-93763633-043f-4fb6-855b-5d646e6cef9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c38342973094c43b412fff9e0f460d6'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:25:38.297053 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-93763633-043f-4fb6-855b-5d646e6cef9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c38342973094c43b412fff9e0f460d6'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:25:38.380317 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-93763633-043f-4fb6-855b-5d646e6cef9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c38342973094c43b412fff9e0f460d6'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:25:38.381651 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-93763633-043f-4fb6-855b-5d646e6cef9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c38342973094c43b412fff9e0f460d6'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:25:38.399576 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1019/2033] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:25:38 2022] DELETE /identity/v3/users/1c38342973094c43b412fff9e0f460d6 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:25:38.822878 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658d679e-02de-4d2a-be64-92ea90ce51b6 service neutron] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:25:38.844162 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658d679e-02de-4d2a-be64-92ea90ce51b6 service neutron] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:25:38.928181 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254590c9-0b4f-4a57-ad22-35462d5a6aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:25:38.928842 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254590c9-0b4f-4a57-ad22-35462d5a6aad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:25:38.929125 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254590c9-0b4f-4a57-ad22-35462d5a6aad admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:25:38.929452 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254590c9-0b4f-4a57-ad22-35462d5a6aad admin admin] PATH_INFO: `/v3/projects/33e38693dbe5490fb39cd1981274a5f6` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:25:38.930633 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254590c9-0b4f-4a57-ad22-35462d5a6aad admin admin] RBAC: Authorizing `identity:delete_project(project_id=33e38693dbe5490fb39cd1981274a5f6)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:25:38.933047 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254590c9-0b4f-4a57-ad22-35462d5a6aad admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:25:39.049776 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-254590c9-0b4f-4a57-ad22-35462d5a6aad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33e38693dbe5490fb39cd1981274a5f6', 'request_id': 'req-254590c9-0b4f-4a57-ad22-35462d5a6aad'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:25:39.056383 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-254590c9-0b4f-4a57-ad22-35462d5a6aad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33e38693dbe5490fb39cd1981274a5f6', 'request_id': 'req-254590c9-0b4f-4a57-ad22-35462d5a6aad'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:25:39.142574 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-254590c9-0b4f-4a57-ad22-35462d5a6aad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33e38693dbe5490fb39cd1981274a5f6', 'request_id': 'req-254590c9-0b4f-4a57-ad22-35462d5a6aad'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:25:39.155957 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-254590c9-0b4f-4a57-ad22-35462d5a6aad admin admin] The token cache is being invalidate because project 33e38693dbe5490fb39cd1981274a5f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81746) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:25:39.156473 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-254590c9-0b4f-4a57-ad22-35462d5a6aad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:25:39.157327 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-254590c9-0b4f-4a57-ad22-35462d5a6aad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:25:39.174978 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93763633-043f-4fb6-855b-5d646e6cef9f admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:25:39.174978 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-93763633-043f-4fb6-855b-5d646e6cef9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:25:39.176614 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1015/2034] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:25:38 2022] DELETE /identity/v3/projects/33e38693dbe5490fb39cd1981274a5f6 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:25:39.176826 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:25:39.182743 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93763633-043f-4fb6-855b-5d646e6cef9f admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:25:39.247921 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d20a037-e65a-4d79-b727-360ae7d68015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:25:39.249248 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d20a037-e65a-4d79-b727-360ae7d68015 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:25:39.249641 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d20a037-e65a-4d79-b727-360ae7d68015 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:25:39.249982 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d20a037-e65a-4d79-b727-360ae7d68015 admin admin] PATH_INFO: `/v3/users/a329566772264f18a91449cf95de5709` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:25:39.251411 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d20a037-e65a-4d79-b727-360ae7d68015 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a329566772264f18a91449cf95de5709)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:25:39.256859 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d20a037-e65a-4d79-b727-360ae7d68015 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:25:39.479702 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-1d20a037-e65a-4d79-b727-360ae7d68015 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a329566772264f18a91449cf95de5709'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:25:39.483508 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-1d20a037-e65a-4d79-b727-360ae7d68015 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a329566772264f18a91449cf95de5709'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:25:39.500937 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-1d20a037-e65a-4d79-b727-360ae7d68015 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a329566772264f18a91449cf95de5709'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:25:39.586641 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-1d20a037-e65a-4d79-b727-360ae7d68015 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a329566772264f18a91449cf95de5709'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:25:39.587575 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-1d20a037-e65a-4d79-b727-360ae7d68015 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a329566772264f18a91449cf95de5709'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:25:39.611388 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1020/2035] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:25:39 2022] DELETE /identity/v3/users/a329566772264f18a91449cf95de5709 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:25:40.349469 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254590c9-0b4f-4a57-ad22-35462d5a6aad admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:25:40.350975 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-254590c9-0b4f-4a57-ad22-35462d5a6aad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:25:40.378237 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254590c9-0b4f-4a57-ad22-35462d5a6aad admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:25:40.465665 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57e0852-7b62-4af1-94d8-cf6d30bdf3a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:25:40.466317 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57e0852-7b62-4af1-94d8-cf6d30bdf3a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:25:40.466597 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57e0852-7b62-4af1-94d8-cf6d30bdf3a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:25:40.466893 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57e0852-7b62-4af1-94d8-cf6d30bdf3a3 admin admin] PATH_INFO: `/v3/projects/a56e1912ce5649e19a687b8f0ee0e566` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:25:40.468147 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57e0852-7b62-4af1-94d8-cf6d30bdf3a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a56e1912ce5649e19a687b8f0ee0e566)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:25:40.471131 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57e0852-7b62-4af1-94d8-cf6d30bdf3a3 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:25:40.627058 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:25:40.639317 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-d57e0852-7b62-4af1-94d8-cf6d30bdf3a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a56e1912ce5649e19a687b8f0ee0e566', 'request_id': 'req-d57e0852-7b62-4af1-94d8-cf6d30bdf3a3'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:25:40.646023 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-d57e0852-7b62-4af1-94d8-cf6d30bdf3a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a56e1912ce5649e19a687b8f0ee0e566', 'request_id': 'req-d57e0852-7b62-4af1-94d8-cf6d30bdf3a3'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:25:40.738790 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-d57e0852-7b62-4af1-94d8-cf6d30bdf3a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a56e1912ce5649e19a687b8f0ee0e566', 'request_id': 'req-d57e0852-7b62-4af1-94d8-cf6d30bdf3a3'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:25:40.752100 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-d57e0852-7b62-4af1-94d8-cf6d30bdf3a3 admin admin] The token cache is being invalidate because project a56e1912ce5649e19a687b8f0ee0e566 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81746) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:25:40.752470 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-d57e0852-7b62-4af1-94d8-cf6d30bdf3a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:25:40.753237 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-d57e0852-7b62-4af1-94d8-cf6d30bdf3a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:25:40.762818 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1016/2036] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:25:40 2022] DELETE /identity/v3/projects/a56e1912ce5649e19a687b8f0ee0e566 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:25:40.851995 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d20a037-e65a-4d79-b727-360ae7d68015 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:25:40.853393 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-1d20a037-e65a-4d79-b727-360ae7d68015 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:25:40.876362 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d20a037-e65a-4d79-b727-360ae7d68015 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:25:40.957681 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8b79b3-b866-4f7a-a29f-c563089dff65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:25:40.958728 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8b79b3-b866-4f7a-a29f-c563089dff65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:25:40.959203 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8b79b3-b866-4f7a-a29f-c563089dff65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:25:40.959696 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8b79b3-b866-4f7a-a29f-c563089dff65 admin admin] PATH_INFO: `/v3/users/cdd87151f3af4a3bb1c3815db5d0e8fa` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:25:40.961169 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8b79b3-b866-4f7a-a29f-c563089dff65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdd87151f3af4a3bb1c3815db5d0e8fa)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:25:40.963774 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8b79b3-b866-4f7a-a29f-c563089dff65 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:25:41.095642 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:25:41.158401 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-ea8b79b3-b866-4f7a-a29f-c563089dff65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdd87151f3af4a3bb1c3815db5d0e8fa'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:25:41.159287 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-ea8b79b3-b866-4f7a-a29f-c563089dff65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdd87151f3af4a3bb1c3815db5d0e8fa'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:25:41.173915 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-ea8b79b3-b866-4f7a-a29f-c563089dff65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdd87151f3af4a3bb1c3815db5d0e8fa'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:25:41.226360 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-ea8b79b3-b866-4f7a-a29f-c563089dff65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdd87151f3af4a3bb1c3815db5d0e8fa'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:25:41.227079 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-ea8b79b3-b866-4f7a-a29f-c563089dff65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdd87151f3af4a3bb1c3815db5d0e8fa'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:25:41.241100 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1021/2037] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:25:40 2022] DELETE /identity/v3/users/cdd87151f3af4a3bb1c3815db5d0e8fa => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:25:41.492101 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57e0852-7b62-4af1-94d8-cf6d30bdf3a3 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:25:41.494614 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-d57e0852-7b62-4af1-94d8-cf6d30bdf3a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:25:41.514479 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57e0852-7b62-4af1-94d8-cf6d30bdf3a3 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:25:41.601948 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853e96ec-ea81-4155-acf7-fdea862b8f51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:25:41.602751 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853e96ec-ea81-4155-acf7-fdea862b8f51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:25:41.603103 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853e96ec-ea81-4155-acf7-fdea862b8f51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:25:41.603439 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853e96ec-ea81-4155-acf7-fdea862b8f51 admin admin] PATH_INFO: `/v3/projects/38c7ea1f3f9848b484a56cf74a2bd129` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:25:41.604630 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853e96ec-ea81-4155-acf7-fdea862b8f51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38c7ea1f3f9848b484a56cf74a2bd129)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:25:41.607038 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853e96ec-ea81-4155-acf7-fdea862b8f51 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:25:41.714613 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-853e96ec-ea81-4155-acf7-fdea862b8f51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38c7ea1f3f9848b484a56cf74a2bd129', 'request_id': 'req-853e96ec-ea81-4155-acf7-fdea862b8f51'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:25:41.722507 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-853e96ec-ea81-4155-acf7-fdea862b8f51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38c7ea1f3f9848b484a56cf74a2bd129', 'request_id': 'req-853e96ec-ea81-4155-acf7-fdea862b8f51'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:25:41.809071 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-853e96ec-ea81-4155-acf7-fdea862b8f51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38c7ea1f3f9848b484a56cf74a2bd129', 'request_id': 'req-853e96ec-ea81-4155-acf7-fdea862b8f51'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:25:41.822041 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-853e96ec-ea81-4155-acf7-fdea862b8f51 admin admin] The token cache is being invalidate because project 38c7ea1f3f9848b484a56cf74a2bd129 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81746) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:25:41.822332 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-853e96ec-ea81-4155-acf7-fdea862b8f51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:25:41.822977 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-853e96ec-ea81-4155-acf7-fdea862b8f51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:25:41.824512 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1017/2038] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:25:41 2022] DELETE /identity/v3/projects/38c7ea1f3f9848b484a56cf74a2bd129 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:25:41.830198 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ced0488-927f-4258-b333-719f5d33d114 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:25:41.830482 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ced0488-927f-4258-b333-719f5d33d114 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:25:41.830758 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ced0488-927f-4258-b333-719f5d33d114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:25:41.852301 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-0ced0488-927f-4258-b333-719f5d33d114 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:25:41.853694 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-0ced0488-927f-4258-b333-719f5d33d114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:25:41.914718 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1022/2039] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:25:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 00:25:41.922418 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e3f9c1-2fcf-4ce2-84e8-65b59d3fbabc None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:25:41.922679 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e3f9c1-2fcf-4ce2-84e8-65b59d3fbabc None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:25:41.922953 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e3f9c1-2fcf-4ce2-84e8-65b59d3fbabc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:25:41.943082 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-a0e3f9c1-2fcf-4ce2-84e8-65b59d3fbabc None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:25:41.944312 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-a0e3f9c1-2fcf-4ce2-84e8-65b59d3fbabc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:25:41.952724 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1018/2040] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:25:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:25:41.957012 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ced0488-927f-4258-b333-719f5d33d114 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:25:41.967673 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ced0488-927f-4258-b333-719f5d33d114 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:25:41.973145 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0496d881-82d5-41de-8194-635eef1a5a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:25:41.973756 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0496d881-82d5-41de-8194-635eef1a5a9d admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:25:41.974032 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0496d881-82d5-41de-8194-635eef1a5a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:25:41.974309 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0496d881-82d5-41de-8194-635eef1a5a9d admin admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:25:41.974953 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0496d881-82d5-41de-8194-635eef1a5a9d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:25:41.975254 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0496d881-82d5-41de-8194-635eef1a5a9d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:25:41.976999 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0496d881-82d5-41de-8194-635eef1a5a9d admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:25:41.989181 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1023/2041] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Oct 12 00:25:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:25:41.994594 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e3f9c1-2fcf-4ce2-84e8-65b59d3fbabc None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:25:42.004158 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e3f9c1-2fcf-4ce2-84e8-65b59d3fbabc None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:25:42.008997 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafb7fe1-c99d-40ec-a1ac-8f28bb2c2d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:25:42.009815 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafb7fe1-c99d-40ec-a1ac-8f28bb2c2d33 admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:25:42.010096 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafb7fe1-c99d-40ec-a1ac-8f28bb2c2d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:25:42.010365 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafb7fe1-c99d-40ec-a1ac-8f28bb2c2d33 admin admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:25:42.010996 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafb7fe1-c99d-40ec-a1ac-8f28bb2c2d33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:25:42.012931 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafb7fe1-c99d-40ec-a1ac-8f28bb2c2d33 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:25:42.088415 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1019/2042] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:25:41 2022] POST /identity/v3/projects => generated 367 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:25:42.092832 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0496d881-82d5-41de-8194-635eef1a5a9d admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:25:42.102552 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0496d881-82d5-41de-8194-635eef1a5a9d admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:25:42.157762 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba99283c-9b8d-44a9-84ce-296513596d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:25:42.158381 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba99283c-9b8d-44a9-84ce-296513596d57 admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:25:42.158670 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba99283c-9b8d-44a9-84ce-296513596d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:25:42.158923 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba99283c-9b8d-44a9-84ce-296513596d57 admin admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:25:42.159592 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba99283c-9b8d-44a9-84ce-296513596d57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:25:42.161502 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba99283c-9b8d-44a9-84ce-296513596d57 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:25:42.265463 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1024/2043] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:25:42 2022] POST /identity/v3/users => generated 284 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 00:25:42.269817 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafb7fe1-c99d-40ec-a1ac-8f28bb2c2d33 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:25:42.280494 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafb7fe1-c99d-40ec-a1ac-8f28bb2c2d33 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:25:42.285368 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0fed71-f0cb-4e92-9516-9ed6dced56ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:25:42.285985 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0fed71-f0cb-4e92-9516-9ed6dced56ac admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:25:42.286267 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0fed71-f0cb-4e92-9516-9ed6dced56ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:25:42.286539 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0fed71-f0cb-4e92-9516-9ed6dced56ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:25:42.287185 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0fed71-f0cb-4e92-9516-9ed6dced56ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:25:42.291181 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0fed71-f0cb-4e92-9516-9ed6dced56ac admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:25:42.302387 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1020/2044] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:25:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:25:42.306296 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba99283c-9b8d-44a9-84ce-296513596d57 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:25:42.316854 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba99283c-9b8d-44a9-84ce-296513596d57 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:25:42.321536 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7585a26e-aa8f-407a-832d-202f87e63362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:25:42.322442 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7585a26e-aa8f-407a-832d-202f87e63362 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:25:42.322442 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7585a26e-aa8f-407a-832d-202f87e63362 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:25:42.322626 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7585a26e-aa8f-407a-832d-202f87e63362 admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:25:42.323260 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7585a26e-aa8f-407a-832d-202f87e63362 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:25:42.325001 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7585a26e-aa8f-407a-832d-202f87e63362 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:25:42.336645 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1025/2045] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:25:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:25:42.340830 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0fed71-f0cb-4e92-9516-9ed6dced56ac admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:25:42.351073 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0fed71-f0cb-4e92-9516-9ed6dced56ac admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:25:42.355666 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ef8419-a146-4afd-8df9-50955e6c1bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:25:42.356372 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef8419-a146-4afd-8df9-50955e6c1bf6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:25:42.356659 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef8419-a146-4afd-8df9-50955e6c1bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:25:42.356934 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef8419-a146-4afd-8df9-50955e6c1bf6 admin admin] PATH_INFO: `/v3/projects/e8365212338946bcad4db05014e030c1/users/db6107095bc8476a8cb5147fa859d74b/roles/ef689f96442945e8a7bc8cbfee13d909` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:25:42.366006 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:25:42.373059 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ef8419-a146-4afd-8df9-50955e6c1bf6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8365212338946bcad4db05014e030c1, user_id=db6107095bc8476a8cb5147fa859d74b, role_id=ef689f96442945e8a7bc8cbfee13d909)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:25:42.376354 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ef8419-a146-4afd-8df9-50955e6c1bf6 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:25:42.490048 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1021/2046] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:25:42 2022] PUT /identity/v3/projects/e8365212338946bcad4db05014e030c1/users/db6107095bc8476a8cb5147fa859d74b/roles/ef689f96442945e8a7bc8cbfee13d909 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:25:42.501915 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb0f6ee-7575-40b2-ab16-5910eb516e9c None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:25:42.502198 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb0f6ee-7575-40b2-ab16-5910eb516e9c None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:25:42.502469 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb0f6ee-7575-40b2-ab16-5910eb516e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:25:42.531087 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:25:42.551948 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-6fb0f6ee-7575-40b2-ab16-5910eb516e9c None None] MFA Rules not processed for user `db6107095bc8476a8cb5147fa859d74b`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:25:42.553238 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-6fb0f6ee-7575-40b2-ab16-5910eb516e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:25:42.626443 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1026/2047] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:25:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:25:42.637297 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ef8419-a146-4afd-8df9-50955e6c1bf6 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:25:42.639471 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-d6ef8419-a146-4afd-8df9-50955e6c1bf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:25:42.704350 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ef8419-a146-4afd-8df9-50955e6c1bf6 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:25:42.708136 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d87487-4a5e-4e21-ad59-ba14b0c92f95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:25:42.708801 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d87487-4a5e-4e21-ad59-ba14b0c92f95 service cinder] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:25:42.709062 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d87487-4a5e-4e21-ad59-ba14b0c92f95 service cinder] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:25:42.709337 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d87487-4a5e-4e21-ad59-ba14b0c92f95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:25:42.718859 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d87487-4a5e-4e21-ad59-ba14b0c92f95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:25:42.720343 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d87487-4a5e-4e21-ad59-ba14b0c92f95 service cinder] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:25:42.729788 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1022/2048] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:25:42 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:25:46.850777 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc2aab9-c6c5-449a-af12-9dd7dec2bd96 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:25:46.851284 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc2aab9-c6c5-449a-af12-9dd7dec2bd96 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:25:46.851427 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc2aab9-c6c5-449a-af12-9dd7dec2bd96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:25:46.875894 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-6dc2aab9-c6c5-449a-af12-9dd7dec2bd96 None None] MFA Rules not processed for user `de766cb582ba4c18b1fda9bf4ad43f65`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:25:46.877182 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-6dc2aab9-c6c5-449a-af12-9dd7dec2bd96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:25:46.936341 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1027/2049] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:25:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:25:52.725473 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=31,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:25:52.877223 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d87487-4a5e-4e21-ad59-ba14b0c92f95 service cinder] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:25:52.938187 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d87487-4a5e-4e21-ad59-ba14b0c92f95 service cinder] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:25:52.942721 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dbf3eb-818b-4816-9fb2-3224a195411b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:25:52.943340 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dbf3eb-818b-4816-9fb2-3224a195411b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:25:52.943644 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dbf3eb-818b-4816-9fb2-3224a195411b admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:25:52.943913 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dbf3eb-818b-4816-9fb2-3224a195411b admin admin] PATH_INFO: `/v3/users/db6107095bc8476a8cb5147fa859d74b` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:25:52.945007 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dbf3eb-818b-4816-9fb2-3224a195411b admin admin] RBAC: Authorizing `identity:delete_user(user_id=db6107095bc8476a8cb5147fa859d74b)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:25:52.946915 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dbf3eb-818b-4816-9fb2-3224a195411b admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:25:53.159309 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-c1dbf3eb-818b-4816-9fb2-3224a195411b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db6107095bc8476a8cb5147fa859d74b'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:25:53.160034 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-c1dbf3eb-818b-4816-9fb2-3224a195411b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db6107095bc8476a8cb5147fa859d74b'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:25:53.172018 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-c1dbf3eb-818b-4816-9fb2-3224a195411b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db6107095bc8476a8cb5147fa859d74b'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:25:53.231143 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-c1dbf3eb-818b-4816-9fb2-3224a195411b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db6107095bc8476a8cb5147fa859d74b'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:25:53.231807 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-c1dbf3eb-818b-4816-9fb2-3224a195411b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db6107095bc8476a8cb5147fa859d74b'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:25:53.244145 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1023/2050] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:25:52 2022] DELETE /identity/v3/users/db6107095bc8476a8cb5147fa859d74b => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:25:53.249088 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad17b2-8fbf-4fc4-b99f-db602d06699e None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:25:53.249565 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad17b2-8fbf-4fc4-b99f-db602d06699e None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:25:53.249655 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad17b2-8fbf-4fc4-b99f-db602d06699e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:25:53.269143 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-74ad17b2-8fbf-4fc4-b99f-db602d06699e None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:25:53.270352 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-74ad17b2-8fbf-4fc4-b99f-db602d06699e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:25:53.329448 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1028/2051] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:25:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:25:53.339204 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dbf3eb-818b-4816-9fb2-3224a195411b admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:25:53.340690 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-c1dbf3eb-818b-4816-9fb2-3224a195411b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:25:53.399733 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dbf3eb-818b-4816-9fb2-3224a195411b admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:25:53.403341 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef175b69-546e-40a9-91d9-0a6b32c0b970 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:25:53.403988 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef175b69-546e-40a9-91d9-0a6b32c0b970 service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:25:53.404311 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef175b69-546e-40a9-91d9-0a6b32c0b970 service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:25:53.404586 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef175b69-546e-40a9-91d9-0a6b32c0b970 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:25:53.414266 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef175b69-546e-40a9-91d9-0a6b32c0b970 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:25:53.415780 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef175b69-546e-40a9-91d9-0a6b32c0b970 service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:25:53.426452 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1024/2052] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:25:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:25:53.850498 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ad17b2-8fbf-4fc4-b99f-db602d06699e None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:25:53.852054 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-74ad17b2-8fbf-4fc4-b99f-db602d06699e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:25:53.866094 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ad17b2-8fbf-4fc4-b99f-db602d06699e None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:25:53.870648 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be5c8f5-d78f-408a-8229-4ac4d6ee8177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:25:53.875966 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be5c8f5-d78f-408a-8229-4ac4d6ee8177 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:25:53.875966 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be5c8f5-d78f-408a-8229-4ac4d6ee8177 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:25:53.875966 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be5c8f5-d78f-408a-8229-4ac4d6ee8177 admin admin] PATH_INFO: `/v3/projects/e8365212338946bcad4db05014e030c1` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:25:53.875966 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be5c8f5-d78f-408a-8229-4ac4d6ee8177 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8365212338946bcad4db05014e030c1)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:25:53.875966 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be5c8f5-d78f-408a-8229-4ac4d6ee8177 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:25:53.962336 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:25:53.969846 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-7be5c8f5-d78f-408a-8229-4ac4d6ee8177 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8365212338946bcad4db05014e030c1', 'request_id': 'req-7be5c8f5-d78f-408a-8229-4ac4d6ee8177'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:25:53.976109 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-7be5c8f5-d78f-408a-8229-4ac4d6ee8177 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8365212338946bcad4db05014e030c1', 'request_id': 'req-7be5c8f5-d78f-408a-8229-4ac4d6ee8177'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:25:54.020342 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-7be5c8f5-d78f-408a-8229-4ac4d6ee8177 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8365212338946bcad4db05014e030c1', 'request_id': 'req-7be5c8f5-d78f-408a-8229-4ac4d6ee8177'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:25:54.031632 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-7be5c8f5-d78f-408a-8229-4ac4d6ee8177 admin admin] The token cache is being invalidate because project e8365212338946bcad4db05014e030c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81747) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:25:54.031914 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-7be5c8f5-d78f-408a-8229-4ac4d6ee8177 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:25:54.032530 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-7be5c8f5-d78f-408a-8229-4ac4d6ee8177 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:25:54.034004 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1029/2053] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:25:53 2022] DELETE /identity/v3/projects/e8365212338946bcad4db05014e030c1 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:25:54.039409 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7c887c-58a6-4f0c-9084-fb2221c619a5 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:25:54.039409 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7c887c-58a6-4f0c-9084-fb2221c619a5 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:25:54.039869 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7c887c-58a6-4f0c-9084-fb2221c619a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:25:54.059899 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-6d7c887c-58a6-4f0c-9084-fb2221c619a5 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:25:54.061110 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-6d7c887c-58a6-4f0c-9084-fb2221c619a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:25:54.079128 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:25:54.119986 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1025/2054] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:25:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:25:54.127654 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ddf941-9dc7-4565-81f1-1e34f5f0a57a None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:25:54.127938 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ddf941-9dc7-4565-81f1-1e34f5f0a57a None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:25:54.130473 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ddf941-9dc7-4565-81f1-1e34f5f0a57a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:25:54.148343 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-24ddf941-9dc7-4565-81f1-1e34f5f0a57a None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:25:54.149579 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-24ddf941-9dc7-4565-81f1-1e34f5f0a57a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:25:54.158027 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1030/2055] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:25:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 00:25:54.162273 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7c887c-58a6-4f0c-9084-fb2221c619a5 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:25:54.171837 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7c887c-58a6-4f0c-9084-fb2221c619a5 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:25:54.176416 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5783ed4c-f9dc-4ad7-8217-6019f0ec5b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:25:54.177018 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5783ed4c-f9dc-4ad7-8217-6019f0ec5b11 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:25:54.177325 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5783ed4c-f9dc-4ad7-8217-6019f0ec5b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:25:54.177596 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5783ed4c-f9dc-4ad7-8217-6019f0ec5b11 admin admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:25:54.178239 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5783ed4c-f9dc-4ad7-8217-6019f0ec5b11 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:25:54.178540 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5783ed4c-f9dc-4ad7-8217-6019f0ec5b11 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:25:54.180198 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5783ed4c-f9dc-4ad7-8217-6019f0ec5b11 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:25:54.191434 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1026/2056] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Oct 12 00:25:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:25:54.195724 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ddf941-9dc7-4565-81f1-1e34f5f0a57a None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:25:54.205231 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ddf941-9dc7-4565-81f1-1e34f5f0a57a None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:25:54.209811 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ccc01a-7faa-47cd-9d79-a89ddc3fd339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:25:54.210394 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ccc01a-7faa-47cd-9d79-a89ddc3fd339 admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:25:54.210664 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ccc01a-7faa-47cd-9d79-a89ddc3fd339 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:25:54.210929 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ccc01a-7faa-47cd-9d79-a89ddc3fd339 admin admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:25:54.211575 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ccc01a-7faa-47cd-9d79-a89ddc3fd339 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:25:54.213504 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ccc01a-7faa-47cd-9d79-a89ddc3fd339 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:25:54.278399 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1031/2057] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:25:54 2022] POST /identity/v3/projects => generated 377 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:25:54.283049 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5783ed4c-f9dc-4ad7-8217-6019f0ec5b11 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:25:54.293163 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5783ed4c-f9dc-4ad7-8217-6019f0ec5b11 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:25:54.348972 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3430c3c4-e0d0-4c97-923e-54aa20280a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:25:54.349576 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3430c3c4-e0d0-4c97-923e-54aa20280a0b admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:25:54.349856 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3430c3c4-e0d0-4c97-923e-54aa20280a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:25:54.350125 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3430c3c4-e0d0-4c97-923e-54aa20280a0b admin admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:25:54.350756 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3430c3c4-e0d0-4c97-923e-54aa20280a0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:25:54.352693 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3430c3c4-e0d0-4c97-923e-54aa20280a0b admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:25:54.442602 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1027/2058] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:25:54 2022] POST /identity/v3/users => generated 289 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:25:54.446659 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ccc01a-7faa-47cd-9d79-a89ddc3fd339 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:25:54.457524 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ccc01a-7faa-47cd-9d79-a89ddc3fd339 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:25:54.462104 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd9df0a-a821-46ce-9259-148d3fdd81ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:25:54.462787 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd9df0a-a821-46ce-9259-148d3fdd81ab admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:25:54.463007 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd9df0a-a821-46ce-9259-148d3fdd81ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:25:54.463276 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd9df0a-a821-46ce-9259-148d3fdd81ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:25:54.463993 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd9df0a-a821-46ce-9259-148d3fdd81ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:25:54.465526 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd9df0a-a821-46ce-9259-148d3fdd81ab admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:25:54.475978 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1032/2059] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:25:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:25:54.479864 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3430c3c4-e0d0-4c97-923e-54aa20280a0b admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:25:54.490084 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3430c3c4-e0d0-4c97-923e-54aa20280a0b admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:25:54.494986 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299b48f8-e447-4d57-acf4-9035d373bc7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:25:54.495579 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299b48f8-e447-4d57-acf4-9035d373bc7b admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:25:54.495966 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299b48f8-e447-4d57-acf4-9035d373bc7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:25:54.496323 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299b48f8-e447-4d57-acf4-9035d373bc7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:25:54.499512 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299b48f8-e447-4d57-acf4-9035d373bc7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:25:54.499512 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299b48f8-e447-4d57-acf4-9035d373bc7b admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:25:54.510229 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1028/2060] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:25:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:25:54.515457 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd9df0a-a821-46ce-9259-148d3fdd81ab admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:25:54.525829 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd9df0a-a821-46ce-9259-148d3fdd81ab admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:25:54.530470 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3940b4b1-579c-43a1-a30c-3e7d653d7f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:25:54.531119 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3940b4b1-579c-43a1-a30c-3e7d653d7f5f admin admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:25:54.531417 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3940b4b1-579c-43a1-a30c-3e7d653d7f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:25:54.531689 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3940b4b1-579c-43a1-a30c-3e7d653d7f5f admin admin] PATH_INFO: `/v3/projects/644a2289407e4856b1e1b3d0fa9429d2/users/6384755476514788855f82cebbc10e71/roles/ef689f96442945e8a7bc8cbfee13d909` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:25:54.546767 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3940b4b1-579c-43a1-a30c-3e7d653d7f5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=644a2289407e4856b1e1b3d0fa9429d2, user_id=6384755476514788855f82cebbc10e71, role_id=ef689f96442945e8a7bc8cbfee13d909)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:25:54.549446 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3940b4b1-579c-43a1-a30c-3e7d653d7f5f admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:25:54.618926 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1033/2061] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:25:54 2022] PUT /identity/v3/projects/644a2289407e4856b1e1b3d0fa9429d2/users/6384755476514788855f82cebbc10e71/roles/ef689f96442945e8a7bc8cbfee13d909 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 00:25:54.631144 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f844965a-63e8-42d1-a78e-f1c237ed80b0 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:25:54.631454 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f844965a-63e8-42d1-a78e-f1c237ed80b0 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:25:54.631728 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f844965a-63e8-42d1-a78e-f1c237ed80b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:25:54.660472 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-f844965a-63e8-42d1-a78e-f1c237ed80b0 None None] MFA Rules not processed for user `6384755476514788855f82cebbc10e71`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:25:54.661665 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-f844965a-63e8-42d1-a78e-f1c237ed80b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:25:54.723520 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1029/2062] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:25:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:25:54.727867 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3940b4b1-579c-43a1-a30c-3e7d653d7f5f admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:25:54.789097 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3940b4b1-579c-43a1-a30c-3e7d653d7f5f admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:25:54.793770 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c776e45-5f6e-4cfa-8a8d-80dbc084277f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:25:54.794369 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c776e45-5f6e-4cfa-8a8d-80dbc084277f admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:25:54.794645 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c776e45-5f6e-4cfa-8a8d-80dbc084277f admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:25:54.794946 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c776e45-5f6e-4cfa-8a8d-80dbc084277f admin admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:25:54.795640 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c776e45-5f6e-4cfa-8a8d-80dbc084277f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:25:54.797516 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c776e45-5f6e-4cfa-8a8d-80dbc084277f admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:25:54.870733 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1034/2063] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:25:54 2022] POST /identity/v3/projects => generated 375 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:25:54.875453 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f844965a-63e8-42d1-a78e-f1c237ed80b0 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:25:54.886380 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f844965a-63e8-42d1-a78e-f1c237ed80b0 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:25:54.898758 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:25:54.941611 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1ae5c6-0bfd-45e1-b877-a2d75b7e553f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:25:54.942216 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ae5c6-0bfd-45e1-b877-a2d75b7e553f admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:25:54.942496 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ae5c6-0bfd-45e1-b877-a2d75b7e553f admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:25:54.942763 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ae5c6-0bfd-45e1-b877-a2d75b7e553f admin admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:25:54.943406 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1ae5c6-0bfd-45e1-b877-a2d75b7e553f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:25:54.945329 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1ae5c6-0bfd-45e1-b877-a2d75b7e553f admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:25:55.052817 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1030/2064] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:25:54 2022] POST /identity/v3/users => generated 294 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 00:25:55.056728 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c776e45-5f6e-4cfa-8a8d-80dbc084277f admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:25:55.067602 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c776e45-5f6e-4cfa-8a8d-80dbc084277f admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:25:55.072205 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b48a539-9036-4592-9d4d-c6dbe40daff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:25:55.072796 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b48a539-9036-4592-9d4d-c6dbe40daff1 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:25:55.073070 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b48a539-9036-4592-9d4d-c6dbe40daff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:25:55.073571 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b48a539-9036-4592-9d4d-c6dbe40daff1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:25:55.074213 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b48a539-9036-4592-9d4d-c6dbe40daff1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:25:55.075882 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b48a539-9036-4592-9d4d-c6dbe40daff1 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:25:55.086147 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1035/2065] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:25:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:25:55.090149 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1ae5c6-0bfd-45e1-b877-a2d75b7e553f admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:25:55.099960 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1ae5c6-0bfd-45e1-b877-a2d75b7e553f admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:25:55.104527 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49543291-a5fd-4687-a307-8934235dd97a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:25:55.105160 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49543291-a5fd-4687-a307-8934235dd97a admin admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:25:55.105434 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49543291-a5fd-4687-a307-8934235dd97a admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:25:55.105733 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49543291-a5fd-4687-a307-8934235dd97a admin admin] PATH_INFO: `/v3/projects/5528d68ffb2d4ea49e7d8214f0db1b2d/users/d2d5872af9cf4584b1ec7f8058bf06b1/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:25:55.120783 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49543291-a5fd-4687-a307-8934235dd97a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5528d68ffb2d4ea49e7d8214f0db1b2d, user_id=d2d5872af9cf4584b1ec7f8058bf06b1, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:25:55.124059 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49543291-a5fd-4687-a307-8934235dd97a admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:25:55.195077 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1031/2066] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:25:55 2022] PUT /identity/v3/projects/5528d68ffb2d4ea49e7d8214f0db1b2d/users/d2d5872af9cf4584b1ec7f8058bf06b1/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:25:55.199050 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b48a539-9036-4592-9d4d-c6dbe40daff1 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:25:55.208998 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b48a539-9036-4592-9d4d-c6dbe40daff1 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:25:55.244061 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:25:55.266471 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5afe74-2277-4914-a77d-ed0a985706dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:25:55.267077 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5afe74-2277-4914-a77d-ed0a985706dc admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:25:55.267352 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5afe74-2277-4914-a77d-ed0a985706dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:25:55.267647 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5afe74-2277-4914-a77d-ed0a985706dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:25:55.268272 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5afe74-2277-4914-a77d-ed0a985706dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:25:55.269888 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5afe74-2277-4914-a77d-ed0a985706dc admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:25:55.280302 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1036/2067] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:25:55 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:25:55.284491 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49543291-a5fd-4687-a307-8934235dd97a admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:25:55.294603 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49543291-a5fd-4687-a307-8934235dd97a admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:25:55.299877 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f97348f-bc52-437a-85c0-8c4b0cde40c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:25:55.300521 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f97348f-bc52-437a-85c0-8c4b0cde40c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:25:55.300798 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f97348f-bc52-437a-85c0-8c4b0cde40c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:25:55.301067 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f97348f-bc52-437a-85c0-8c4b0cde40c5 admin admin] PATH_INFO: `/v3/projects/5528d68ffb2d4ea49e7d8214f0db1b2d/users/d2d5872af9cf4584b1ec7f8058bf06b1/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:25:55.303030 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f97348f-bc52-437a-85c0-8c4b0cde40c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5528d68ffb2d4ea49e7d8214f0db1b2d, user_id=d2d5872af9cf4584b1ec7f8058bf06b1, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:25:55.305624 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f97348f-bc52-437a-85c0-8c4b0cde40c5 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:25:55.315895 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1032/2068] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:25:55 2022] PUT /identity/v3/projects/5528d68ffb2d4ea49e7d8214f0db1b2d/users/d2d5872af9cf4584b1ec7f8058bf06b1/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:25:55.325904 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d912205c-4629-4559-9e77-4bf6e4c34942 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:25:55.326189 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d912205c-4629-4559-9e77-4bf6e4c34942 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:25:55.326468 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d912205c-4629-4559-9e77-4bf6e4c34942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:25:55.356436 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-d912205c-4629-4559-9e77-4bf6e4c34942 None None] MFA Rules not processed for user `d2d5872af9cf4584b1ec7f8058bf06b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:25:55.358229 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-d912205c-4629-4559-9e77-4bf6e4c34942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:25:55.430382 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1037/2069] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:25:55 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:25:55.441084 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f97348f-bc52-437a-85c0-8c4b0cde40c5 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:25:55.442811 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-3f97348f-bc52-437a-85c0-8c4b0cde40c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:25:55.503714 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f97348f-bc52-437a-85c0-8c4b0cde40c5 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:25:55.507428 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0bf32e-7a28-4146-963b-59bb26d7d780 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:25:55.508044 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0bf32e-7a28-4146-963b-59bb26d7d780 service cinder] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:25:55.508323 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0bf32e-7a28-4146-963b-59bb26d7d780 service cinder] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:25:55.508649 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0bf32e-7a28-4146-963b-59bb26d7d780 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:25:55.567710 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0bf32e-7a28-4146-963b-59bb26d7d780 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:25:55.569081 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0bf32e-7a28-4146-963b-59bb26d7d780 service cinder] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:25:55.578647 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1033/2070] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:25:55 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:25:57.599652 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d912205c-4629-4559-9e77-4bf6e4c34942 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:25:57.602703 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-d912205c-4629-4559-9e77-4bf6e4c34942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:25:57.677955 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d912205c-4629-4559-9e77-4bf6e4c34942 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:25:57.684019 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444d6104-4c5f-49b9-8436-a17ab64a52a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a515b75aa3944912ad2364ffd7924f40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:25:57.684644 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d6104-4c5f-49b9-8436-a17ab64a52a4 service nova] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:25:57.684987 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d6104-4c5f-49b9-8436-a17ab64a52a4 service nova] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:25:57.685265 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d6104-4c5f-49b9-8436-a17ab64a52a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:25:57.687864 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-444d6104-4c5f-49b9-8436-a17ab64a52a4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:25:57.702788 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444d6104-4c5f-49b9-8436-a17ab64a52a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:25:57.704987 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444d6104-4c5f-49b9-8436-a17ab64a52a4 service nova] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:25:57.718614 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1038/2071] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:25:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:26:01.845147 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0bf32e-7a28-4146-963b-59bb26d7d780 service cinder] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:26:01.846558 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-bf0bf32e-7a28-4146-963b-59bb26d7d780 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:26:01.857916 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0bf32e-7a28-4146-963b-59bb26d7d780 service cinder] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:26:01.861587 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef3c487-0855-4d97-9bcd-2588b4c5950f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:26:01.862178 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef3c487-0855-4d97-9bcd-2588b4c5950f service cinder] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:26:01.862454 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef3c487-0855-4d97-9bcd-2588b4c5950f service cinder] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:26:01.862723 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef3c487-0855-4d97-9bcd-2588b4c5950f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:26:01.871632 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef3c487-0855-4d97-9bcd-2588b4c5950f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:26:01.873056 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef3c487-0855-4d97-9bcd-2588b4c5950f service cinder] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:26:01.882287 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1034/2072] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:26:01 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:26:07.712314 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=80 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:26:11.878150 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=90,keystone:INSERT=5 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:26:29.003376 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444d6104-4c5f-49b9-8436-a17ab64a52a4 service nova] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:26:29.066068 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444d6104-4c5f-49b9-8436-a17ab64a52a4 service nova] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:26:29.070558 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb64283-32c1-47ea-9d14-6d2cbfae3ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:26:29.071192 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb64283-32c1-47ea-9d14-6d2cbfae3ca1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:26:29.071501 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb64283-32c1-47ea-9d14-6d2cbfae3ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:26:29.071774 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb64283-32c1-47ea-9d14-6d2cbfae3ca1 admin admin] PATH_INFO: `/v3/users/6384755476514788855f82cebbc10e71` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:26:29.072893 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb64283-32c1-47ea-9d14-6d2cbfae3ca1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6384755476514788855f82cebbc10e71)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:26:29.074914 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb64283-32c1-47ea-9d14-6d2cbfae3ca1 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:26:29.303225 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-5eb64283-32c1-47ea-9d14-6d2cbfae3ca1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6384755476514788855f82cebbc10e71'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:26:29.304099 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-5eb64283-32c1-47ea-9d14-6d2cbfae3ca1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6384755476514788855f82cebbc10e71'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:26:29.316420 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-5eb64283-32c1-47ea-9d14-6d2cbfae3ca1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6384755476514788855f82cebbc10e71'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:26:29.374589 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-5eb64283-32c1-47ea-9d14-6d2cbfae3ca1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6384755476514788855f82cebbc10e71'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:26:29.375221 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-5eb64283-32c1-47ea-9d14-6d2cbfae3ca1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6384755476514788855f82cebbc10e71'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:26:29.388818 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1039/2073] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:26:29 2022] DELETE /identity/v3/users/6384755476514788855f82cebbc10e71 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:26:29.393601 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeeefce-b1a3-40e2-ad54-245d4eb46278 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:26:29.394060 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeeefce-b1a3-40e2-ad54-245d4eb46278 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:26:29.394186 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeeefce-b1a3-40e2-ad54-245d4eb46278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:26:29.412840 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-baeeefce-b1a3-40e2-ad54-245d4eb46278 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:26:29.414048 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-baeeefce-b1a3-40e2-ad54-245d4eb46278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:26:29.472613 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1035/2074] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:26:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:26:29.482481 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb64283-32c1-47ea-9d14-6d2cbfae3ca1 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:26:29.483913 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-5eb64283-32c1-47ea-9d14-6d2cbfae3ca1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:26:29.542154 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb64283-32c1-47ea-9d14-6d2cbfae3ca1 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:26:29.546105 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3861fdf-dc00-4558-bbae-288202377632 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:26:29.546783 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3861fdf-dc00-4558-bbae-288202377632 service neutron] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:26:29.547058 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3861fdf-dc00-4558-bbae-288202377632 service neutron] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:26:29.547328 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3861fdf-dc00-4558-bbae-288202377632 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:26:29.557622 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3861fdf-dc00-4558-bbae-288202377632 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:26:29.559052 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3861fdf-dc00-4558-bbae-288202377632 service neutron] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:26:29.569372 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1040/2075] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:26:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:26:30.005160 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeeefce-b1a3-40e2-ad54-245d4eb46278 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:26:30.006663 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-baeeefce-b1a3-40e2-ad54-245d4eb46278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:26:30.020416 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeeefce-b1a3-40e2-ad54-245d4eb46278 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:26:30.025574 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b5eb7f-15b4-4a09-8416-dbf785e25cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:26:30.026212 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b5eb7f-15b4-4a09-8416-dbf785e25cb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:26:30.026494 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b5eb7f-15b4-4a09-8416-dbf785e25cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:26:30.027462 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b5eb7f-15b4-4a09-8416-dbf785e25cb5 admin admin] PATH_INFO: `/v3/projects/644a2289407e4856b1e1b3d0fa9429d2` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:26:30.028398 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b5eb7f-15b4-4a09-8416-dbf785e25cb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=644a2289407e4856b1e1b3d0fa9429d2)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:26:30.031267 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b5eb7f-15b4-4a09-8416-dbf785e25cb5 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:26:30.163268 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-66b5eb7f-15b4-4a09-8416-dbf785e25cb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '644a2289407e4856b1e1b3d0fa9429d2', 'request_id': 'req-66b5eb7f-15b4-4a09-8416-dbf785e25cb5'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:26:30.169598 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-66b5eb7f-15b4-4a09-8416-dbf785e25cb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '644a2289407e4856b1e1b3d0fa9429d2', 'request_id': 'req-66b5eb7f-15b4-4a09-8416-dbf785e25cb5'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:26:30.241780 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-66b5eb7f-15b4-4a09-8416-dbf785e25cb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '644a2289407e4856b1e1b3d0fa9429d2', 'request_id': 'req-66b5eb7f-15b4-4a09-8416-dbf785e25cb5'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:26:30.252430 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-66b5eb7f-15b4-4a09-8416-dbf785e25cb5 admin admin] The token cache is being invalidate because project 644a2289407e4856b1e1b3d0fa9429d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81746) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:26:30.252719 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-66b5eb7f-15b4-4a09-8416-dbf785e25cb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:26:30.253340 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-66b5eb7f-15b4-4a09-8416-dbf785e25cb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:26:30.254834 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1036/2076] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:26:30 2022] DELETE /identity/v3/projects/644a2289407e4856b1e1b3d0fa9429d2 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:26:30.258822 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3861fdf-dc00-4558-bbae-288202377632 service neutron] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:26:30.269213 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3861fdf-dc00-4558-bbae-288202377632 service neutron] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:26:30.287622 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:26:30.329015 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10fd5fa-44e0-4e5e-aafd-2c92abbe5beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:26:30.329752 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10fd5fa-44e0-4e5e-aafd-2c92abbe5beb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:26:30.330029 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10fd5fa-44e0-4e5e-aafd-2c92abbe5beb admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:26:30.330803 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10fd5fa-44e0-4e5e-aafd-2c92abbe5beb admin admin] PATH_INFO: `/v3/users/d2d5872af9cf4584b1ec7f8058bf06b1` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:26:30.332100 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10fd5fa-44e0-4e5e-aafd-2c92abbe5beb admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2d5872af9cf4584b1ec7f8058bf06b1)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:26:30.334123 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10fd5fa-44e0-4e5e-aafd-2c92abbe5beb admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:26:30.515518 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-a10fd5fa-44e0-4e5e-aafd-2c92abbe5beb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2d5872af9cf4584b1ec7f8058bf06b1'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:26:30.516198 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-a10fd5fa-44e0-4e5e-aafd-2c92abbe5beb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2d5872af9cf4584b1ec7f8058bf06b1'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:26:30.530998 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-a10fd5fa-44e0-4e5e-aafd-2c92abbe5beb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2d5872af9cf4584b1ec7f8058bf06b1'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:26:30.586261 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-a10fd5fa-44e0-4e5e-aafd-2c92abbe5beb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2d5872af9cf4584b1ec7f8058bf06b1'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:26:30.587781 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-a10fd5fa-44e0-4e5e-aafd-2c92abbe5beb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2d5872af9cf4584b1ec7f8058bf06b1'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:26:30.600936 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1041/2077] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:26:30 2022] DELETE /identity/v3/users/d2d5872af9cf4584b1ec7f8058bf06b1 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:26:30.988291 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b5eb7f-15b4-4a09-8416-dbf785e25cb5 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:26:30.989703 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-66b5eb7f-15b4-4a09-8416-dbf785e25cb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:26:31.003974 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b5eb7f-15b4-4a09-8416-dbf785e25cb5 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:26:31.065308 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5d1572-a02c-42af-ad27-84d9a862b8ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:26:31.065789 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5d1572-a02c-42af-ad27-84d9a862b8ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:26:31.066069 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5d1572-a02c-42af-ad27-84d9a862b8ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:26:31.066341 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5d1572-a02c-42af-ad27-84d9a862b8ed admin admin] PATH_INFO: `/v3/projects/5528d68ffb2d4ea49e7d8214f0db1b2d` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:26:31.067571 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5d1572-a02c-42af-ad27-84d9a862b8ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=5528d68ffb2d4ea49e7d8214f0db1b2d)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:26:31.069487 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5d1572-a02c-42af-ad27-84d9a862b8ed admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:26:31.159469 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:26:31.175733 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-fa5d1572-a02c-42af-ad27-84d9a862b8ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5528d68ffb2d4ea49e7d8214f0db1b2d', 'request_id': 'req-fa5d1572-a02c-42af-ad27-84d9a862b8ed'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:26:31.181995 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-fa5d1572-a02c-42af-ad27-84d9a862b8ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5528d68ffb2d4ea49e7d8214f0db1b2d', 'request_id': 'req-fa5d1572-a02c-42af-ad27-84d9a862b8ed'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:26:31.277345 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-fa5d1572-a02c-42af-ad27-84d9a862b8ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5528d68ffb2d4ea49e7d8214f0db1b2d', 'request_id': 'req-fa5d1572-a02c-42af-ad27-84d9a862b8ed'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:26:31.289688 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-fa5d1572-a02c-42af-ad27-84d9a862b8ed admin admin] The token cache is being invalidate because project 5528d68ffb2d4ea49e7d8214f0db1b2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81746) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:26:31.290080 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-fa5d1572-a02c-42af-ad27-84d9a862b8ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:26:31.290791 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-fa5d1572-a02c-42af-ad27-84d9a862b8ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:26:31.292562 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1037/2078] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:26:30 2022] DELETE /identity/v3/projects/5528d68ffb2d4ea49e7d8214f0db1b2d => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:26:31.297760 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafa3cb7-dfb1-4fd0-80a5-0c20b1779c53 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:26:31.298227 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafa3cb7-dfb1-4fd0-80a5-0c20b1779c53 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:26:31.298320 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafa3cb7-dfb1-4fd0-80a5-0c20b1779c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:26:31.318361 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-aafa3cb7-dfb1-4fd0-80a5-0c20b1779c53 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:26:31.319603 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-aafa3cb7-dfb1-4fd0-80a5-0c20b1779c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:26:31.378609 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1042/2079] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:26:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:26:31.386467 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff17a5b-30b0-49b1-823f-72b5c0f8ebcb None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:26:31.386750 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff17a5b-30b0-49b1-823f-72b5c0f8ebcb None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:26:31.387020 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff17a5b-30b0-49b1-823f-72b5c0f8ebcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:26:31.406633 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-9ff17a5b-30b0-49b1-823f-72b5c0f8ebcb None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:26:31.407857 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-9ff17a5b-30b0-49b1-823f-72b5c0f8ebcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:26:31.416180 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1038/2080] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:26:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:26:31.420394 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafa3cb7-dfb1-4fd0-80a5-0c20b1779c53 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:26:31.430703 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafa3cb7-dfb1-4fd0-80a5-0c20b1779c53 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:26:31.436868 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8397f6c8-51f8-4d52-8ab4-3a4c829a311b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:26:31.436868 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8397f6c8-51f8-4d52-8ab4-3a4c829a311b admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:26:31.436868 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8397f6c8-51f8-4d52-8ab4-3a4c829a311b admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:26:31.436868 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8397f6c8-51f8-4d52-8ab4-3a4c829a311b admin admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:26:31.437320 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8397f6c8-51f8-4d52-8ab4-3a4c829a311b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:26:31.437320 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8397f6c8-51f8-4d52-8ab4-3a4c829a311b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:26:31.438880 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8397f6c8-51f8-4d52-8ab4-3a4c829a311b admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:26:31.450227 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1043/2081] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Oct 12 00:26:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:26:31.456103 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff17a5b-30b0-49b1-823f-72b5c0f8ebcb None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:26:31.465051 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff17a5b-30b0-49b1-823f-72b5c0f8ebcb None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:26:31.469593 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c67be64-02ee-49c6-b3e3-f772d563f3a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:26:31.470180 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c67be64-02ee-49c6-b3e3-f772d563f3a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:26:31.470453 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c67be64-02ee-49c6-b3e3-f772d563f3a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:26:31.470723 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c67be64-02ee-49c6-b3e3-f772d563f3a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:26:31.471892 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c67be64-02ee-49c6-b3e3-f772d563f3a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:26:31.473807 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c67be64-02ee-49c6-b3e3-f772d563f3a0 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:26:31.553417 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1039/2082] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:26:31 2022] POST /identity/v3/projects => generated 385 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:26:31.557587 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8397f6c8-51f8-4d52-8ab4-3a4c829a311b admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:26:31.574879 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8397f6c8-51f8-4d52-8ab4-3a4c829a311b admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:26:31.593826 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:26:31.656743 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2dcd6e-9cd7-4be7-a4c9-2821a6802908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:26:31.657387 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2dcd6e-9cd7-4be7-a4c9-2821a6802908 admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:26:31.657661 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2dcd6e-9cd7-4be7-a4c9-2821a6802908 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:26:31.657977 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2dcd6e-9cd7-4be7-a4c9-2821a6802908 admin admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:26:31.659218 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2dcd6e-9cd7-4be7-a4c9-2821a6802908 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:26:31.661344 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2dcd6e-9cd7-4be7-a4c9-2821a6802908 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:26:31.741200 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1044/2083] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:26:31 2022] POST /identity/v3/users => generated 293 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:26:31.748500 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c67be64-02ee-49c6-b3e3-f772d563f3a0 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:26:31.782952 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c67be64-02ee-49c6-b3e3-f772d563f3a0 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:26:31.788265 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1144bbe-1a27-4a6a-bae7-39b39faaa23e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:26:31.788866 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1144bbe-1a27-4a6a-bae7-39b39faaa23e admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:26:31.789140 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1144bbe-1a27-4a6a-bae7-39b39faaa23e admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:26:31.789409 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1144bbe-1a27-4a6a-bae7-39b39faaa23e admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:26:31.790042 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1144bbe-1a27-4a6a-bae7-39b39faaa23e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:26:31.791659 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1144bbe-1a27-4a6a-bae7-39b39faaa23e admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:26:31.802271 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1040/2084] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:26:31 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:26:31.806117 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2dcd6e-9cd7-4be7-a4c9-2821a6802908 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:26:31.816062 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2dcd6e-9cd7-4be7-a4c9-2821a6802908 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:26:31.820671 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8f8b35-ce31-4d7f-a9c9-ed54b4adfb31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:26:31.821260 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8f8b35-ce31-4d7f-a9c9-ed54b4adfb31 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:26:31.821551 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8f8b35-ce31-4d7f-a9c9-ed54b4adfb31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:26:31.821798 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8f8b35-ce31-4d7f-a9c9-ed54b4adfb31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:26:31.822423 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8f8b35-ce31-4d7f-a9c9-ed54b4adfb31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:26:31.824039 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8f8b35-ce31-4d7f-a9c9-ed54b4adfb31 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:26:31.835116 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1045/2085] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:26:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 00:26:31.839112 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1144bbe-1a27-4a6a-bae7-39b39faaa23e admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:26:31.849629 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1144bbe-1a27-4a6a-bae7-39b39faaa23e admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:26:31.854200 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b48af0-2faa-4684-9206-68808480781e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:26:31.854837 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b48af0-2faa-4684-9206-68808480781e admin admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:26:31.855113 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b48af0-2faa-4684-9206-68808480781e admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:26:31.855405 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b48af0-2faa-4684-9206-68808480781e admin admin] PATH_INFO: `/v3/projects/dd0b5fd22fad45e49d5d110fa58946f7/users/88251b7e2cfd42be8f54e5b3304d6d6b/roles/ef689f96442945e8a7bc8cbfee13d909` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:26:31.869563 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b48af0-2faa-4684-9206-68808480781e admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd0b5fd22fad45e49d5d110fa58946f7, user_id=88251b7e2cfd42be8f54e5b3304d6d6b, role_id=ef689f96442945e8a7bc8cbfee13d909)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:26:31.872236 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b48af0-2faa-4684-9206-68808480781e admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:26:31.957428 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1041/2086] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:26:31 2022] PUT /identity/v3/projects/dd0b5fd22fad45e49d5d110fa58946f7/users/88251b7e2cfd42be8f54e5b3304d6d6b/roles/ef689f96442945e8a7bc8cbfee13d909 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:26:31.967632 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06d31a3-a87a-4d18-ba27-3463946dbb0c None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:26:31.967910 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06d31a3-a87a-4d18-ba27-3463946dbb0c None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:26:31.968175 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06d31a3-a87a-4d18-ba27-3463946dbb0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:26:32.005370 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-c06d31a3-a87a-4d18-ba27-3463946dbb0c None None] MFA Rules not processed for user `88251b7e2cfd42be8f54e5b3304d6d6b`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:26:32.006605 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-c06d31a3-a87a-4d18-ba27-3463946dbb0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:26:32.069999 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1046/2087] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:26:31 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:26:32.074466 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b48af0-2faa-4684-9206-68808480781e admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:26:32.136983 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b48af0-2faa-4684-9206-68808480781e admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:26:32.141918 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5995a17-29a8-4a48-961b-cf4b670481d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:26:32.142525 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5995a17-29a8-4a48-961b-cf4b670481d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:26:32.143456 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5995a17-29a8-4a48-961b-cf4b670481d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:26:32.143740 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5995a17-29a8-4a48-961b-cf4b670481d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:26:32.144361 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5995a17-29a8-4a48-961b-cf4b670481d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:26:32.146239 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5995a17-29a8-4a48-961b-cf4b670481d6 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:26:32.250574 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1042/2088] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:26:32 2022] POST /identity/v3/projects => generated 387 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:26:32.255391 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06d31a3-a87a-4d18-ba27-3463946dbb0c None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:26:32.266360 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06d31a3-a87a-4d18-ba27-3463946dbb0c None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:26:32.317244 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:26:32.323919 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a19fac0-55da-4135-98df-928e1ba09f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:26:32.324514 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a19fac0-55da-4135-98df-928e1ba09f2a admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:26:32.324792 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a19fac0-55da-4135-98df-928e1ba09f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:26:32.325059 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a19fac0-55da-4135-98df-928e1ba09f2a admin admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:26:32.325689 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a19fac0-55da-4135-98df-928e1ba09f2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:26:32.327693 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a19fac0-55da-4135-98df-928e1ba09f2a admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:26:32.409177 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1047/2089] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:26:32 2022] POST /identity/v3/users => generated 300 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:26:32.413434 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5995a17-29a8-4a48-961b-cf4b670481d6 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:26:32.423736 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5995a17-29a8-4a48-961b-cf4b670481d6 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:26:32.428815 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a65e4b6-ca77-45ec-85d9-64f996d0aa07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:26:32.429445 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a65e4b6-ca77-45ec-85d9-64f996d0aa07 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:26:32.429722 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a65e4b6-ca77-45ec-85d9-64f996d0aa07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:26:32.429988 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a65e4b6-ca77-45ec-85d9-64f996d0aa07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:26:32.430617 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a65e4b6-ca77-45ec-85d9-64f996d0aa07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:26:32.432238 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a65e4b6-ca77-45ec-85d9-64f996d0aa07 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:26:32.437370 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:26:32.443785 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1043/2090] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:26:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:26:32.447968 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a19fac0-55da-4135-98df-928e1ba09f2a admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:26:32.467133 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a19fac0-55da-4135-98df-928e1ba09f2a admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:26:32.471929 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5560667e-0eef-4ee8-87ce-4298ea9108fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:26:32.472576 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5560667e-0eef-4ee8-87ce-4298ea9108fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:26:32.472852 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5560667e-0eef-4ee8-87ce-4298ea9108fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:26:32.473124 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5560667e-0eef-4ee8-87ce-4298ea9108fc admin admin] PATH_INFO: `/v3/projects/bb6fd8fec0bf4df4b94ca643a453bd85/users/01a9d6b4af8640aba2f0213306b75fa8/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:26:32.488668 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5560667e-0eef-4ee8-87ce-4298ea9108fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb6fd8fec0bf4df4b94ca643a453bd85, user_id=01a9d6b4af8640aba2f0213306b75fa8, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:26:32.491738 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5560667e-0eef-4ee8-87ce-4298ea9108fc admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:26:32.609263 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1048/2091] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:26:32 2022] PUT /identity/v3/projects/bb6fd8fec0bf4df4b94ca643a453bd85/users/01a9d6b4af8640aba2f0213306b75fa8/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:26:32.613881 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a65e4b6-ca77-45ec-85d9-64f996d0aa07 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:26:32.633648 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a65e4b6-ca77-45ec-85d9-64f996d0aa07 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:26:32.688077 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f0530f-50c3-4749-be07-1835558b0c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:26:32.688946 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f0530f-50c3-4749-be07-1835558b0c4a admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:26:32.689498 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f0530f-50c3-4749-be07-1835558b0c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:26:32.689793 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f0530f-50c3-4749-be07-1835558b0c4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:26:32.690876 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f0530f-50c3-4749-be07-1835558b0c4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:26:32.692542 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f0530f-50c3-4749-be07-1835558b0c4a admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:26:32.702114 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1044/2092] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:26:32 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 00:26:32.706360 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5560667e-0eef-4ee8-87ce-4298ea9108fc admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:26:32.716708 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5560667e-0eef-4ee8-87ce-4298ea9108fc admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:26:32.722079 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2efaefb-f04f-4820-95d3-0727cc4b3951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:26:32.722797 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2efaefb-f04f-4820-95d3-0727cc4b3951 admin admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:26:32.723153 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2efaefb-f04f-4820-95d3-0727cc4b3951 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:26:32.723521 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2efaefb-f04f-4820-95d3-0727cc4b3951 admin admin] PATH_INFO: `/v3/projects/bb6fd8fec0bf4df4b94ca643a453bd85/users/01a9d6b4af8640aba2f0213306b75fa8/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:26:32.725468 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2efaefb-f04f-4820-95d3-0727cc4b3951 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb6fd8fec0bf4df4b94ca643a453bd85, user_id=01a9d6b4af8640aba2f0213306b75fa8, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:26:32.728185 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2efaefb-f04f-4820-95d3-0727cc4b3951 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:26:32.739592 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1049/2093] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:26:32 2022] PUT /identity/v3/projects/bb6fd8fec0bf4df4b94ca643a453bd85/users/01a9d6b4af8640aba2f0213306b75fa8/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:26:32.749990 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e26c1c0-f755-4e68-ba52-c60cad08af7f None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:26:32.750280 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e26c1c0-f755-4e68-ba52-c60cad08af7f None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:26:32.750559 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e26c1c0-f755-4e68-ba52-c60cad08af7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:26:32.778669 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-8e26c1c0-f755-4e68-ba52-c60cad08af7f None None] MFA Rules not processed for user `01a9d6b4af8640aba2f0213306b75fa8`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:26:32.779934 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-8e26c1c0-f755-4e68-ba52-c60cad08af7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:26:32.847745 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1045/2094] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:26:32 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:26:32.858954 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2efaefb-f04f-4820-95d3-0727cc4b3951 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:26:32.860763 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-a2efaefb-f04f-4820-95d3-0727cc4b3951 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:26:32.965178 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2efaefb-f04f-4820-95d3-0727cc4b3951 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:26:32.969119 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbea749-6083-4907-abbd-e4a244bcc288 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:26:32.970121 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbea749-6083-4907-abbd-e4a244bcc288 service cinder] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:26:32.970121 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbea749-6083-4907-abbd-e4a244bcc288 service cinder] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:26:32.970315 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbea749-6083-4907-abbd-e4a244bcc288 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:26:32.980500 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbea749-6083-4907-abbd-e4a244bcc288 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:26:32.981924 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbea749-6083-4907-abbd-e4a244bcc288 service cinder] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:26:32.992041 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1050/2095] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:26:32 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:26:33.022678 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e26c1c0-f755-4e68-ba52-c60cad08af7f None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:26:33.085987 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e26c1c0-f755-4e68-ba52-c60cad08af7f None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:26:33.090595 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ec151b-a3c0-43a7-b5df-a8c8756f683e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:26:33.091226 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ec151b-a3c0-43a7-b5df-a8c8756f683e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:26:33.091540 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ec151b-a3c0-43a7-b5df-a8c8756f683e admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:26:33.091815 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ec151b-a3c0-43a7-b5df-a8c8756f683e admin admin] PATH_INFO: `/v3/users/88251b7e2cfd42be8f54e5b3304d6d6b` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:26:33.092944 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ec151b-a3c0-43a7-b5df-a8c8756f683e admin admin] RBAC: Authorizing `identity:delete_user(user_id=88251b7e2cfd42be8f54e5b3304d6d6b)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:26:33.094850 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ec151b-a3c0-43a7-b5df-a8c8756f683e admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:26:33.109815 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:26:33.268832 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-a4ec151b-a3c0-43a7-b5df-a8c8756f683e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88251b7e2cfd42be8f54e5b3304d6d6b'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:26:33.269485 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-a4ec151b-a3c0-43a7-b5df-a8c8756f683e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88251b7e2cfd42be8f54e5b3304d6d6b'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:26:33.281310 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-a4ec151b-a3c0-43a7-b5df-a8c8756f683e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88251b7e2cfd42be8f54e5b3304d6d6b'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:26:33.357153 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-a4ec151b-a3c0-43a7-b5df-a8c8756f683e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88251b7e2cfd42be8f54e5b3304d6d6b'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:26:33.357794 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-a4ec151b-a3c0-43a7-b5df-a8c8756f683e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88251b7e2cfd42be8f54e5b3304d6d6b'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:26:33.370161 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1046/2096] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:26:33 2022] DELETE /identity/v3/users/88251b7e2cfd42be8f54e5b3304d6d6b => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:26:33.374631 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e27eabe-c328-4670-999f-471b54386928 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:26:33.375109 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e27eabe-c328-4670-999f-471b54386928 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:26:33.375213 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e27eabe-c328-4670-999f-471b54386928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:26:33.403471 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-9e27eabe-c328-4670-999f-471b54386928 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:26:33.404595 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-9e27eabe-c328-4670-999f-471b54386928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:26:33.448152 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:26:33.463711 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1051/2097] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:26:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:26:33.473924 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ec151b-a3c0-43a7-b5df-a8c8756f683e admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:26:33.475313 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-a4ec151b-a3c0-43a7-b5df-a8c8756f683e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:26:33.552832 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ec151b-a3c0-43a7-b5df-a8c8756f683e admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:26:33.556522 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc3dac9-51c5-474c-80bf-7ebf2b658cf3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:26:33.557126 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc3dac9-51c5-474c-80bf-7ebf2b658cf3 service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:26:33.557400 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc3dac9-51c5-474c-80bf-7ebf2b658cf3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:26:33.557669 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc3dac9-51c5-474c-80bf-7ebf2b658cf3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:26:33.566415 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc3dac9-51c5-474c-80bf-7ebf2b658cf3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:26:33.567928 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc3dac9-51c5-474c-80bf-7ebf2b658cf3 service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:26:33.577496 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1047/2098] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:26:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 00:26:34.011454 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e27eabe-c328-4670-999f-471b54386928 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:26:34.011454 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-9e27eabe-c328-4670-999f-471b54386928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:26:34.026405 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e27eabe-c328-4670-999f-471b54386928 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:26:34.030940 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eb1ead-0ef1-4fb6-9b30-0404433b9080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:26:34.031627 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb1ead-0ef1-4fb6-9b30-0404433b9080 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:26:34.031930 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb1ead-0ef1-4fb6-9b30-0404433b9080 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:26:34.032208 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb1ead-0ef1-4fb6-9b30-0404433b9080 admin admin] PATH_INFO: `/v3/projects/dd0b5fd22fad45e49d5d110fa58946f7` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:26:34.033381 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eb1ead-0ef1-4fb6-9b30-0404433b9080 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd0b5fd22fad45e49d5d110fa58946f7)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:26:34.035595 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eb1ead-0ef1-4fb6-9b30-0404433b9080 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:26:34.149695 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-c1eb1ead-0ef1-4fb6-9b30-0404433b9080 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd0b5fd22fad45e49d5d110fa58946f7', 'request_id': 'req-c1eb1ead-0ef1-4fb6-9b30-0404433b9080'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:26:34.155892 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-c1eb1ead-0ef1-4fb6-9b30-0404433b9080 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd0b5fd22fad45e49d5d110fa58946f7', 'request_id': 'req-c1eb1ead-0ef1-4fb6-9b30-0404433b9080'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:26:34.224467 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-c1eb1ead-0ef1-4fb6-9b30-0404433b9080 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd0b5fd22fad45e49d5d110fa58946f7', 'request_id': 'req-c1eb1ead-0ef1-4fb6-9b30-0404433b9080'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:26:34.235644 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-c1eb1ead-0ef1-4fb6-9b30-0404433b9080 admin admin] The token cache is being invalidate because project dd0b5fd22fad45e49d5d110fa58946f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81747) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:26:34.235820 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-c1eb1ead-0ef1-4fb6-9b30-0404433b9080 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:26:34.236490 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-c1eb1ead-0ef1-4fb6-9b30-0404433b9080 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:26:34.238594 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1052/2099] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:26:34 2022] DELETE /identity/v3/projects/dd0b5fd22fad45e49d5d110fa58946f7 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:26:34.242684 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc3dac9-51c5-474c-80bf-7ebf2b658cf3 service neutron] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:26:34.252242 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc3dac9-51c5-474c-80bf-7ebf2b658cf3 service neutron] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:26:34.307946 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e03f56-9caa-41d6-b489-6ddf98149d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:26:34.308827 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e03f56-9caa-41d6-b489-6ddf98149d32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:26:34.309099 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e03f56-9caa-41d6-b489-6ddf98149d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:26:34.309364 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e03f56-9caa-41d6-b489-6ddf98149d32 admin admin] PATH_INFO: `/v3/users/01a9d6b4af8640aba2f0213306b75fa8` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:26:34.310468 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e03f56-9caa-41d6-b489-6ddf98149d32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01a9d6b4af8640aba2f0213306b75fa8)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:26:34.312407 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e03f56-9caa-41d6-b489-6ddf98149d32 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:26:34.328165 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:26:34.513765 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-39e03f56-9caa-41d6-b489-6ddf98149d32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01a9d6b4af8640aba2f0213306b75fa8'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:26:34.514354 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-39e03f56-9caa-41d6-b489-6ddf98149d32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01a9d6b4af8640aba2f0213306b75fa8'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:26:34.526772 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-39e03f56-9caa-41d6-b489-6ddf98149d32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01a9d6b4af8640aba2f0213306b75fa8'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:26:34.584555 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-39e03f56-9caa-41d6-b489-6ddf98149d32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01a9d6b4af8640aba2f0213306b75fa8'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:26:34.585165 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-39e03f56-9caa-41d6-b489-6ddf98149d32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01a9d6b4af8640aba2f0213306b75fa8'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:26:34.598011 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1048/2100] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:26:34 2022] DELETE /identity/v3/users/01a9d6b4af8640aba2f0213306b75fa8 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:26:35.028200 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eb1ead-0ef1-4fb6-9b30-0404433b9080 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:26:35.029648 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-c1eb1ead-0ef1-4fb6-9b30-0404433b9080 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:26:35.043711 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eb1ead-0ef1-4fb6-9b30-0404433b9080 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:26:35.107583 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bdf9f0-a55c-491b-ae59-c597aeefdea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:26:35.108217 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bdf9f0-a55c-491b-ae59-c597aeefdea5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:26:35.108495 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bdf9f0-a55c-491b-ae59-c597aeefdea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:26:35.108770 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bdf9f0-a55c-491b-ae59-c597aeefdea5 admin admin] PATH_INFO: `/v3/projects/bb6fd8fec0bf4df4b94ca643a453bd85` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:26:35.109878 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bdf9f0-a55c-491b-ae59-c597aeefdea5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb6fd8fec0bf4df4b94ca643a453bd85)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:26:35.111817 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bdf9f0-a55c-491b-ae59-c597aeefdea5 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:26:35.213224 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-75bdf9f0-a55c-491b-ae59-c597aeefdea5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb6fd8fec0bf4df4b94ca643a453bd85', 'request_id': 'req-75bdf9f0-a55c-491b-ae59-c597aeefdea5'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:26:35.219529 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-75bdf9f0-a55c-491b-ae59-c597aeefdea5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb6fd8fec0bf4df4b94ca643a453bd85', 'request_id': 'req-75bdf9f0-a55c-491b-ae59-c597aeefdea5'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:26:35.293249 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-75bdf9f0-a55c-491b-ae59-c597aeefdea5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb6fd8fec0bf4df4b94ca643a453bd85', 'request_id': 'req-75bdf9f0-a55c-491b-ae59-c597aeefdea5'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:26:35.303917 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-75bdf9f0-a55c-491b-ae59-c597aeefdea5 admin admin] The token cache is being invalidate because project bb6fd8fec0bf4df4b94ca643a453bd85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81747) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:26:35.304200 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-75bdf9f0-a55c-491b-ae59-c597aeefdea5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:26:35.304817 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-75bdf9f0-a55c-491b-ae59-c597aeefdea5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:26:35.306308 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1053/2101] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:26:35 2022] DELETE /identity/v3/projects/bb6fd8fec0bf4df4b94ca643a453bd85 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:26:35.311189 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947811b8-3d10-4ab6-aee8-de5059da4cd4 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:26:35.313659 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947811b8-3d10-4ab6-aee8-de5059da4cd4 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:26:35.313659 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947811b8-3d10-4ab6-aee8-de5059da4cd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:26:35.341125 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-947811b8-3d10-4ab6-aee8-de5059da4cd4 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:26:35.342310 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-947811b8-3d10-4ab6-aee8-de5059da4cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:26:35.399086 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1049/2102] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:26:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:26:35.408010 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f19fefd-1f22-4901-98c5-6372d1324777 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:26:35.408295 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f19fefd-1f22-4901-98c5-6372d1324777 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:26:35.408572 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f19fefd-1f22-4901-98c5-6372d1324777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:26:35.416758 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:26:35.428956 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-2f19fefd-1f22-4901-98c5-6372d1324777 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:26:35.430493 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-2f19fefd-1f22-4901-98c5-6372d1324777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:26:35.438197 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1054/2103] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:26:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:26:35.442519 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947811b8-3d10-4ab6-aee8-de5059da4cd4 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:26:35.452358 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947811b8-3d10-4ab6-aee8-de5059da4cd4 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:26:35.457578 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba92d82c-7247-4b6c-b829-a0f9cb6bee01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:26:35.458327 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba92d82c-7247-4b6c-b829-a0f9cb6bee01 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:26:35.458503 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba92d82c-7247-4b6c-b829-a0f9cb6bee01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:26:35.458778 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba92d82c-7247-4b6c-b829-a0f9cb6bee01 admin admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:26:35.459460 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba92d82c-7247-4b6c-b829-a0f9cb6bee01 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:26:35.459762 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba92d82c-7247-4b6c-b829-a0f9cb6bee01 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:26:35.461356 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba92d82c-7247-4b6c-b829-a0f9cb6bee01 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:26:35.473435 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1050/2104] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Oct 12 00:26:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:26:35.477714 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f19fefd-1f22-4901-98c5-6372d1324777 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:26:35.487026 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f19fefd-1f22-4901-98c5-6372d1324777 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:26:35.491620 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e6e3ea-d81d-41f2-93b5-2b38f41c010e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:26:35.492241 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e6e3ea-d81d-41f2-93b5-2b38f41c010e admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:26:35.492486 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e6e3ea-d81d-41f2-93b5-2b38f41c010e admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:26:35.492755 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e6e3ea-d81d-41f2-93b5-2b38f41c010e admin admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:26:35.493368 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e6e3ea-d81d-41f2-93b5-2b38f41c010e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:26:35.495256 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e6e3ea-d81d-41f2-93b5-2b38f41c010e admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:26:35.588784 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1055/2105] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:26:35 2022] POST /identity/v3/projects => generated 407 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:26:35.592930 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba92d82c-7247-4b6c-b829-a0f9cb6bee01 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:26:35.603900 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba92d82c-7247-4b6c-b829-a0f9cb6bee01 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:26:35.659967 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f0c3fe-60c2-4e5b-b4f9-733d6dffb850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:26:35.660569 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f0c3fe-60c2-4e5b-b4f9-733d6dffb850 admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:26:35.660848 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f0c3fe-60c2-4e5b-b4f9-733d6dffb850 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:26:35.661119 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f0c3fe-60c2-4e5b-b4f9-733d6dffb850 admin admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:26:35.661754 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f0c3fe-60c2-4e5b-b4f9-733d6dffb850 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:26:35.663695 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f0c3fe-60c2-4e5b-b4f9-733d6dffb850 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:26:35.734461 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1051/2106] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:26:35 2022] POST /identity/v3/users => generated 304 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:26:35.738635 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e6e3ea-d81d-41f2-93b5-2b38f41c010e admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:26:35.749065 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e6e3ea-d81d-41f2-93b5-2b38f41c010e admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:26:35.754244 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bb15be-b97e-4d88-8eaf-846c301eb3fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:26:35.754831 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb15be-b97e-4d88-8eaf-846c301eb3fe admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:26:35.755103 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb15be-b97e-4d88-8eaf-846c301eb3fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:26:35.755395 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb15be-b97e-4d88-8eaf-846c301eb3fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:26:35.756031 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bb15be-b97e-4d88-8eaf-846c301eb3fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:26:35.757642 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bb15be-b97e-4d88-8eaf-846c301eb3fe admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:26:35.767640 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1056/2107] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:26:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:26:35.772267 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f0c3fe-60c2-4e5b-b4f9-733d6dffb850 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:26:35.777422 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:26:35.782770 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f0c3fe-60c2-4e5b-b4f9-733d6dffb850 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:26:35.787404 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a2a6fc-6f0e-4800-aeda-702aa73b8da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:26:35.787998 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a2a6fc-6f0e-4800-aeda-702aa73b8da8 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:26:35.788276 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a2a6fc-6f0e-4800-aeda-702aa73b8da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:26:35.788546 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a2a6fc-6f0e-4800-aeda-702aa73b8da8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:26:35.789179 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a2a6fc-6f0e-4800-aeda-702aa73b8da8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:26:35.790860 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a2a6fc-6f0e-4800-aeda-702aa73b8da8 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:26:35.801205 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1052/2108] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:26:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:26:35.805158 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bb15be-b97e-4d88-8eaf-846c301eb3fe admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:26:35.814935 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bb15be-b97e-4d88-8eaf-846c301eb3fe admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:26:35.819963 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5d15eb-dc08-49a2-a336-a1dc78767891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:26:35.820541 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5d15eb-dc08-49a2-a336-a1dc78767891 admin admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:26:35.820814 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5d15eb-dc08-49a2-a336-a1dc78767891 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:26:35.821076 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5d15eb-dc08-49a2-a336-a1dc78767891 admin admin] PATH_INFO: `/v3/projects/0a8dd4e57c4e43208ebce66e500978f1/users/6bc3079027f24d459e8d53695985806a/roles/ef689f96442945e8a7bc8cbfee13d909` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:26:35.835920 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5d15eb-dc08-49a2-a336-a1dc78767891 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a8dd4e57c4e43208ebce66e500978f1, user_id=6bc3079027f24d459e8d53695985806a, role_id=ef689f96442945e8a7bc8cbfee13d909)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:26:35.838530 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5d15eb-dc08-49a2-a336-a1dc78767891 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:26:35.929014 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1057/2109] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:26:35 2022] PUT /identity/v3/projects/0a8dd4e57c4e43208ebce66e500978f1/users/6bc3079027f24d459e8d53695985806a/roles/ef689f96442945e8a7bc8cbfee13d909 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:26:35.939161 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f615fc09-22a0-4fee-bcd7-e42d2fd5c501 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:26:35.939478 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f615fc09-22a0-4fee-bcd7-e42d2fd5c501 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:26:35.939753 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f615fc09-22a0-4fee-bcd7-e42d2fd5c501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:26:35.967741 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-f615fc09-22a0-4fee-bcd7-e42d2fd5c501 None None] MFA Rules not processed for user `6bc3079027f24d459e8d53695985806a`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:26:35.968935 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-f615fc09-22a0-4fee-bcd7-e42d2fd5c501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:26:36.031114 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1053/2110] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:26:35 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 00:26:36.035571 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5d15eb-dc08-49a2-a336-a1dc78767891 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:26:36.097601 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5d15eb-dc08-49a2-a336-a1dc78767891 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:26:36.102172 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e8b2de-7acd-4c3f-a1a2-ef64a3e7bf2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:26:36.102772 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e8b2de-7acd-4c3f-a1a2-ef64a3e7bf2f admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:26:36.103047 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e8b2de-7acd-4c3f-a1a2-ef64a3e7bf2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:26:36.103317 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e8b2de-7acd-4c3f-a1a2-ef64a3e7bf2f admin admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:26:36.103978 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e8b2de-7acd-4c3f-a1a2-ef64a3e7bf2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:26:36.106099 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e8b2de-7acd-4c3f-a1a2-ef64a3e7bf2f admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:26:36.204844 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1058/2111] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:26:36 2022] POST /identity/v3/projects => generated 407 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:26:36.208829 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f615fc09-22a0-4fee-bcd7-e42d2fd5c501 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:26:36.219568 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f615fc09-22a0-4fee-bcd7-e42d2fd5c501 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:26:36.275913 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49a72e8-2859-410b-a511-da3c7b73e8e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:26:36.276552 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49a72e8-2859-410b-a511-da3c7b73e8e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:26:36.276826 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49a72e8-2859-410b-a511-da3c7b73e8e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:26:36.277094 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49a72e8-2859-410b-a511-da3c7b73e8e0 admin admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:26:36.277727 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49a72e8-2859-410b-a511-da3c7b73e8e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:26:36.279706 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49a72e8-2859-410b-a511-da3c7b73e8e0 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:26:36.335008 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1054/2112] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:26:36 2022] POST /identity/v3/users => generated 310 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:26:36.338586 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e8b2de-7acd-4c3f-a1a2-ef64a3e7bf2f admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:26:36.349131 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e8b2de-7acd-4c3f-a1a2-ef64a3e7bf2f admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:26:36.353818 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ba002e-f7b1-4097-b015-35a213134765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:26:36.354416 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ba002e-f7b1-4097-b015-35a213134765 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:26:36.354691 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ba002e-f7b1-4097-b015-35a213134765 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:26:36.354962 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ba002e-f7b1-4097-b015-35a213134765 admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:26:36.355656 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ba002e-f7b1-4097-b015-35a213134765 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:26:36.357287 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ba002e-f7b1-4097-b015-35a213134765 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:26:36.368215 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1059/2113] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:26:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:26:36.372333 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49a72e8-2859-410b-a511-da3c7b73e8e0 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:26:36.382993 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49a72e8-2859-410b-a511-da3c7b73e8e0 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:26:36.387508 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfba55d1-3466-477e-b3e5-a0bbbcd90fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:26:36.388144 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba55d1-3466-477e-b3e5-a0bbbcd90fa8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:26:36.388420 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba55d1-3466-477e-b3e5-a0bbbcd90fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:26:36.388690 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba55d1-3466-477e-b3e5-a0bbbcd90fa8 admin admin] PATH_INFO: `/v3/projects/7ebba2fbe77c4f349026cba57f461b7e/users/180c640d8f4548b1aa9eb5a07c9d17b1/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:26:36.404324 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfba55d1-3466-477e-b3e5-a0bbbcd90fa8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ebba2fbe77c4f349026cba57f461b7e, user_id=180c640d8f4548b1aa9eb5a07c9d17b1, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:26:36.408065 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfba55d1-3466-477e-b3e5-a0bbbcd90fa8 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:26:36.477360 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1055/2114] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:26:36 2022] PUT /identity/v3/projects/7ebba2fbe77c4f349026cba57f461b7e/users/180c640d8f4548b1aa9eb5a07c9d17b1/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:26:36.481240 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ba002e-f7b1-4097-b015-35a213134765 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:26:36.492083 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ba002e-f7b1-4097-b015-35a213134765 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:26:36.526071 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfba55d1-3466-477e-b3e5-a0bbbcd90fa8 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:26:36.527855 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-dfba55d1-3466-477e-b3e5-a0bbbcd90fa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:26:36.552179 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed5e79a-8c7c-46d0-b1b6-c13d07ae5701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:26:36.553184 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed5e79a-8c7c-46d0-b1b6-c13d07ae5701 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:26:36.553305 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed5e79a-8c7c-46d0-b1b6-c13d07ae5701 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:26:36.553570 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed5e79a-8c7c-46d0-b1b6-c13d07ae5701 admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:26:36.554206 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed5e79a-8c7c-46d0-b1b6-c13d07ae5701 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:26:36.555957 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed5e79a-8c7c-46d0-b1b6-c13d07ae5701 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:26:36.566174 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1060/2115] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:26:36 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:26:36.570330 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed5e79a-8c7c-46d0-b1b6-c13d07ae5701 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:26:36.580589 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed5e79a-8c7c-46d0-b1b6-c13d07ae5701 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:26:36.582430 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:26:36.585567 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658bf23d-5c50-4719-89f8-8a4166a16666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:26:36.586150 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658bf23d-5c50-4719-89f8-8a4166a16666 admin admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:26:36.586415 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658bf23d-5c50-4719-89f8-8a4166a16666 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:26:36.586684 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658bf23d-5c50-4719-89f8-8a4166a16666 admin admin] PATH_INFO: `/v3/projects/7ebba2fbe77c4f349026cba57f461b7e/users/180c640d8f4548b1aa9eb5a07c9d17b1/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:26:36.588582 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658bf23d-5c50-4719-89f8-8a4166a16666 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ebba2fbe77c4f349026cba57f461b7e, user_id=180c640d8f4548b1aa9eb5a07c9d17b1, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:26:36.591178 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658bf23d-5c50-4719-89f8-8a4166a16666 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:26:36.594226 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:26:36.602003 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1061/2116] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:26:36 2022] PUT /identity/v3/projects/7ebba2fbe77c4f349026cba57f461b7e/users/180c640d8f4548b1aa9eb5a07c9d17b1/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 00:26:36.607072 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfba55d1-3466-477e-b3e5-a0bbbcd90fa8 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:26:36.611895 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe30cf9-8b26-4ea7-941e-6b961fce5e2e None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:26:36.612714 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe30cf9-8b26-4ea7-941e-6b961fce5e2e None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:26:36.612987 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe30cf9-8b26-4ea7-941e-6b961fce5e2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:26:36.641182 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-0fe30cf9-8b26-4ea7-941e-6b961fce5e2e None None] MFA Rules not processed for user `180c640d8f4548b1aa9eb5a07c9d17b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:26:36.643455 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-0fe30cf9-8b26-4ea7-941e-6b961fce5e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:26:36.672580 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc80534-d2e3-47c9-9dfb-88ee591bab04 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a515b75aa3944912ad2364ffd7924f40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:26:36.673196 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc80534-d2e3-47c9-9dfb-88ee591bab04 service nova] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:26:36.673475 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc80534-d2e3-47c9-9dfb-88ee591bab04 service nova] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:26:36.673747 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc80534-d2e3-47c9-9dfb-88ee591bab04 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:26:36.676124 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-acc80534-d2e3-47c9-9dfb-88ee591bab04 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:26:36.712621 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1062/2117] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:26:36 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 00:26:36.723402 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe30cf9-8b26-4ea7-941e-6b961fce5e2e None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:26:36.724729 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-0fe30cf9-8b26-4ea7-941e-6b961fce5e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:26:36.735132 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc80534-d2e3-47c9-9dfb-88ee591bab04 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:26:36.736781 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc80534-d2e3-47c9-9dfb-88ee591bab04 service nova] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:26:36.746794 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1056/2118] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:26:36 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 00:26:36.781471 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe30cf9-8b26-4ea7-941e-6b961fce5e2e None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:26:36.785886 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f8e132-a8f1-41c1-9655-38f79af00920 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:26:36.786541 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f8e132-a8f1-41c1-9655-38f79af00920 service cinder] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:26:36.786819 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f8e132-a8f1-41c1-9655-38f79af00920 service cinder] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:26:36.787147 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f8e132-a8f1-41c1-9655-38f79af00920 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:26:36.796787 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f8e132-a8f1-41c1-9655-38f79af00920 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:26:36.798019 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f8e132-a8f1-41c1-9655-38f79af00920 service cinder] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:26:36.808579 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1063/2119] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:26:36 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 00:26:37.208756 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc80534-d2e3-47c9-9dfb-88ee591bab04 service nova] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:26:37.217983 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc80534-d2e3-47c9-9dfb-88ee591bab04 service nova] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:26:37.221584 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf130fb-d732-419e-b5ef-a416c910ae24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:26:37.222176 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf130fb-d732-419e-b5ef-a416c910ae24 service cinder] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:26:37.222457 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf130fb-d732-419e-b5ef-a416c910ae24 service cinder] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:26:37.222728 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf130fb-d732-419e-b5ef-a416c910ae24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:26:37.278517 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf130fb-d732-419e-b5ef-a416c910ae24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:26:37.280001 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf130fb-d732-419e-b5ef-a416c910ae24 service cinder] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:26:37.288983 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1057/2120] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:26:37 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 00:26:46.803522 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=62 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:26:46.954517 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f8e132-a8f1-41c1-9655-38f79af00920 service cinder] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:26:47.015645 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f8e132-a8f1-41c1-9655-38f79af00920 service cinder] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:26:47.020816 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a80ccf2-56b3-49fc-be09-83f2a93160af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:26:47.021431 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a80ccf2-56b3-49fc-be09-83f2a93160af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:26:47.021705 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a80ccf2-56b3-49fc-be09-83f2a93160af admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:26:47.021972 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a80ccf2-56b3-49fc-be09-83f2a93160af admin admin] PATH_INFO: `/v3/users/6bc3079027f24d459e8d53695985806a` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:26:47.023060 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a80ccf2-56b3-49fc-be09-83f2a93160af admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bc3079027f24d459e8d53695985806a)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:26:47.024993 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a80ccf2-56b3-49fc-be09-83f2a93160af admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:26:47.163892 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-9a80ccf2-56b3-49fc-be09-83f2a93160af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bc3079027f24d459e8d53695985806a'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:26:47.164545 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-9a80ccf2-56b3-49fc-be09-83f2a93160af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bc3079027f24d459e8d53695985806a'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:26:47.176491 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-9a80ccf2-56b3-49fc-be09-83f2a93160af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bc3079027f24d459e8d53695985806a'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:26:47.259172 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-9a80ccf2-56b3-49fc-be09-83f2a93160af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bc3079027f24d459e8d53695985806a'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:26:47.259816 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-9a80ccf2-56b3-49fc-be09-83f2a93160af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bc3079027f24d459e8d53695985806a'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:26:47.273516 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1064/2121] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:26:46 2022] DELETE /identity/v3/users/6bc3079027f24d459e8d53695985806a => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:26:47.277954 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20dfe97-98f4-46d8-9d6f-1d075f8b88b9 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:26:47.278417 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20dfe97-98f4-46d8-9d6f-1d075f8b88b9 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:26:47.278531 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20dfe97-98f4-46d8-9d6f-1d075f8b88b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:26:47.297623 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-a20dfe97-98f4-46d8-9d6f-1d075f8b88b9 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:26:47.298853 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-a20dfe97-98f4-46d8-9d6f-1d075f8b88b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:26:47.312247 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:26:47.358892 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1058/2122] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:26:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:26:47.368551 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a80ccf2-56b3-49fc-be09-83f2a93160af admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:26:47.372309 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-9a80ccf2-56b3-49fc-be09-83f2a93160af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:26:47.426575 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a80ccf2-56b3-49fc-be09-83f2a93160af admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:26:47.430700 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9feb71-a70c-497a-a89d-0fdccf210e6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:26:47.431306 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9feb71-a70c-497a-a89d-0fdccf210e6a service neutron] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:26:47.431608 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9feb71-a70c-497a-a89d-0fdccf210e6a service neutron] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:26:47.431874 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9feb71-a70c-497a-a89d-0fdccf210e6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:26:47.441086 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9feb71-a70c-497a-a89d-0fdccf210e6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:26:47.443082 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9feb71-a70c-497a-a89d-0fdccf210e6a service neutron] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:26:47.452562 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1065/2123] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:26:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:26:47.931488 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20dfe97-98f4-46d8-9d6f-1d075f8b88b9 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:26:47.933421 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-a20dfe97-98f4-46d8-9d6f-1d075f8b88b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:26:47.947282 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20dfe97-98f4-46d8-9d6f-1d075f8b88b9 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:26:47.959003 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f983edb6-4e04-40c1-94e0-816c3cc55107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:26:47.959003 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f983edb6-4e04-40c1-94e0-816c3cc55107 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:26:47.959003 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f983edb6-4e04-40c1-94e0-816c3cc55107 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:26:47.959003 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f983edb6-4e04-40c1-94e0-816c3cc55107 admin admin] PATH_INFO: `/v3/projects/0a8dd4e57c4e43208ebce66e500978f1` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:26:47.960017 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f983edb6-4e04-40c1-94e0-816c3cc55107 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a8dd4e57c4e43208ebce66e500978f1)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:26:47.961901 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f983edb6-4e04-40c1-94e0-816c3cc55107 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:26:48.089507 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-f983edb6-4e04-40c1-94e0-816c3cc55107 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a8dd4e57c4e43208ebce66e500978f1', 'request_id': 'req-f983edb6-4e04-40c1-94e0-816c3cc55107'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:26:48.096107 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-f983edb6-4e04-40c1-94e0-816c3cc55107 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a8dd4e57c4e43208ebce66e500978f1', 'request_id': 'req-f983edb6-4e04-40c1-94e0-816c3cc55107'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:26:48.184788 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-f983edb6-4e04-40c1-94e0-816c3cc55107 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a8dd4e57c4e43208ebce66e500978f1', 'request_id': 'req-f983edb6-4e04-40c1-94e0-816c3cc55107'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:26:48.196410 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-f983edb6-4e04-40c1-94e0-816c3cc55107 admin admin] The token cache is being invalidate because project 0a8dd4e57c4e43208ebce66e500978f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81746) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:26:48.196723 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-f983edb6-4e04-40c1-94e0-816c3cc55107 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:26:48.197375 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-f983edb6-4e04-40c1-94e0-816c3cc55107 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:26:48.199095 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1059/2124] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:26:47 2022] DELETE /identity/v3/projects/0a8dd4e57c4e43208ebce66e500978f1 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:26:48.203082 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9feb71-a70c-497a-a89d-0fdccf210e6a service neutron] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:26:48.212908 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9feb71-a70c-497a-a89d-0fdccf210e6a service neutron] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:26:48.243457 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:26:48.282015 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06656254-6c8a-49ad-bd90-ea931667f214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:26:48.282710 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06656254-6c8a-49ad-bd90-ea931667f214 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:26:48.282984 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06656254-6c8a-49ad-bd90-ea931667f214 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:26:48.283255 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06656254-6c8a-49ad-bd90-ea931667f214 admin admin] PATH_INFO: `/v3/users/180c640d8f4548b1aa9eb5a07c9d17b1` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:26:48.284408 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06656254-6c8a-49ad-bd90-ea931667f214 admin admin] RBAC: Authorizing `identity:delete_user(user_id=180c640d8f4548b1aa9eb5a07c9d17b1)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:26:48.286905 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06656254-6c8a-49ad-bd90-ea931667f214 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:26:48.617176 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-06656254-6c8a-49ad-bd90-ea931667f214 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '180c640d8f4548b1aa9eb5a07c9d17b1'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:26:48.617815 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-06656254-6c8a-49ad-bd90-ea931667f214 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '180c640d8f4548b1aa9eb5a07c9d17b1'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:26:48.630012 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-06656254-6c8a-49ad-bd90-ea931667f214 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '180c640d8f4548b1aa9eb5a07c9d17b1'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:26:48.712073 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-06656254-6c8a-49ad-bd90-ea931667f214 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '180c640d8f4548b1aa9eb5a07c9d17b1'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:26:48.712722 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-06656254-6c8a-49ad-bd90-ea931667f214 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '180c640d8f4548b1aa9eb5a07c9d17b1'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:26:48.726125 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1066/2125] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:26:48 2022] DELETE /identity/v3/users/180c640d8f4548b1aa9eb5a07c9d17b1 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:26:49.238988 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f983edb6-4e04-40c1-94e0-816c3cc55107 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:26:49.240486 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-f983edb6-4e04-40c1-94e0-816c3cc55107 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:26:49.256116 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f983edb6-4e04-40c1-94e0-816c3cc55107 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:26:49.313683 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c45c46-e141-4db5-8ca2-fe60503d405b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:26:49.314321 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c45c46-e141-4db5-8ca2-fe60503d405b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:26:49.314601 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c45c46-e141-4db5-8ca2-fe60503d405b admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:26:49.314873 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c45c46-e141-4db5-8ca2-fe60503d405b admin admin] PATH_INFO: `/v3/projects/7ebba2fbe77c4f349026cba57f461b7e` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:26:49.316082 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c45c46-e141-4db5-8ca2-fe60503d405b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ebba2fbe77c4f349026cba57f461b7e)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:26:49.337309 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c45c46-e141-4db5-8ca2-fe60503d405b admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:26:49.423524 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:26:49.426171 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-f2c45c46-e141-4db5-8ca2-fe60503d405b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ebba2fbe77c4f349026cba57f461b7e', 'request_id': 'req-f2c45c46-e141-4db5-8ca2-fe60503d405b'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:26:49.432641 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-f2c45c46-e141-4db5-8ca2-fe60503d405b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ebba2fbe77c4f349026cba57f461b7e', 'request_id': 'req-f2c45c46-e141-4db5-8ca2-fe60503d405b'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:26:49.511720 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-f2c45c46-e141-4db5-8ca2-fe60503d405b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ebba2fbe77c4f349026cba57f461b7e', 'request_id': 'req-f2c45c46-e141-4db5-8ca2-fe60503d405b'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:26:49.523670 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-f2c45c46-e141-4db5-8ca2-fe60503d405b admin admin] The token cache is being invalidate because project 7ebba2fbe77c4f349026cba57f461b7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81746) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:26:49.523965 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-f2c45c46-e141-4db5-8ca2-fe60503d405b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:26:49.524843 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-f2c45c46-e141-4db5-8ca2-fe60503d405b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:26:49.527161 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1060/2126] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:26:49 2022] DELETE /identity/v3/projects/7ebba2fbe77c4f349026cba57f461b7e => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:26:49.538863 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2872610a-5e53-4f82-aabd-e95118bae898 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:26:49.539350 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2872610a-5e53-4f82-aabd-e95118bae898 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:26:49.539500 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2872610a-5e53-4f82-aabd-e95118bae898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:26:49.564563 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-2872610a-5e53-4f82-aabd-e95118bae898 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:26:49.565753 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-2872610a-5e53-4f82-aabd-e95118bae898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:26:49.626854 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1067/2127] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:26:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:26:49.636423 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8880046-68aa-403d-a942-d1ec75b3ec05 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:26:49.636964 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8880046-68aa-403d-a942-d1ec75b3ec05 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:26:49.637389 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8880046-68aa-403d-a942-d1ec75b3ec05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:26:49.658068 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-b8880046-68aa-403d-a942-d1ec75b3ec05 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:26:49.659206 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-b8880046-68aa-403d-a942-d1ec75b3ec05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:26:49.667157 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1061/2128] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:26:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:26:49.671322 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2872610a-5e53-4f82-aabd-e95118bae898 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:26:49.680932 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2872610a-5e53-4f82-aabd-e95118bae898 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:26:49.685492 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11405983-d8ba-45e1-a141-4567f6fe7ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:26:49.686108 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11405983-d8ba-45e1-a141-4567f6fe7ea2 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:26:49.686382 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11405983-d8ba-45e1-a141-4567f6fe7ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:26:49.686646 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11405983-d8ba-45e1-a141-4567f6fe7ea2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:26:49.687279 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11405983-d8ba-45e1-a141-4567f6fe7ea2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:26:49.687759 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11405983-d8ba-45e1-a141-4567f6fe7ea2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:26:49.689336 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11405983-d8ba-45e1-a141-4567f6fe7ea2 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:26:49.701184 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1068/2129] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Oct 12 00:26:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:26:49.705521 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8880046-68aa-403d-a942-d1ec75b3ec05 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:26:49.715272 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8880046-68aa-403d-a942-d1ec75b3ec05 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:26:49.719960 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7007a60-7df6-4266-9b23-f706e7bb5ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:26:49.720554 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7007a60-7df6-4266-9b23-f706e7bb5ea3 admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:26:49.720831 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7007a60-7df6-4266-9b23-f706e7bb5ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:26:49.721099 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7007a60-7df6-4266-9b23-f706e7bb5ea3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:26:49.721748 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7007a60-7df6-4266-9b23-f706e7bb5ea3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:26:49.723738 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7007a60-7df6-4266-9b23-f706e7bb5ea3 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:26:49.787085 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1062/2130] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:26:49 2022] POST /identity/v3/projects => generated 391 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:26:49.791301 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11405983-d8ba-45e1-a141-4567f6fe7ea2 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:26:49.801364 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11405983-d8ba-45e1-a141-4567f6fe7ea2 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:26:49.827602 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:26:49.858265 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4aee78-a7d8-4a21-9033-1c741790d9e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:26:49.859054 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4aee78-a7d8-4a21-9033-1c741790d9e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:26:49.859540 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4aee78-a7d8-4a21-9033-1c741790d9e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:26:49.860144 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4aee78-a7d8-4a21-9033-1c741790d9e1 admin admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:26:49.860817 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4aee78-a7d8-4a21-9033-1c741790d9e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:26:49.862746 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4aee78-a7d8-4a21-9033-1c741790d9e1 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:26:49.950652 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1069/2131] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:26:49 2022] POST /identity/v3/users => generated 296 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:26:49.951973 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7007a60-7df6-4266-9b23-f706e7bb5ea3 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:26:49.962465 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7007a60-7df6-4266-9b23-f706e7bb5ea3 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:26:49.967201 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70e0663-f76e-4d13-8adf-60719afc2a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:26:49.967876 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e0663-f76e-4d13-8adf-60719afc2a10 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:26:49.968153 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e0663-f76e-4d13-8adf-60719afc2a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:26:49.968427 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e0663-f76e-4d13-8adf-60719afc2a10 admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:26:49.969077 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70e0663-f76e-4d13-8adf-60719afc2a10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:26:49.970732 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70e0663-f76e-4d13-8adf-60719afc2a10 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:26:49.981106 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1063/2132] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:26:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:26:49.985142 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4aee78-a7d8-4a21-9033-1c741790d9e1 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:26:49.995993 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4aee78-a7d8-4a21-9033-1c741790d9e1 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:26:50.000842 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65f594c-31cc-4f3f-9075-cd71fa6ae8e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:26:50.001622 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65f594c-31cc-4f3f-9075-cd71fa6ae8e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:26:50.001823 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65f594c-31cc-4f3f-9075-cd71fa6ae8e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:26:50.002103 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65f594c-31cc-4f3f-9075-cd71fa6ae8e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:26:50.002776 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65f594c-31cc-4f3f-9075-cd71fa6ae8e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:26:50.005108 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65f594c-31cc-4f3f-9075-cd71fa6ae8e8 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:26:50.018522 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1070/2133] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:26:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:26:50.019781 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70e0663-f76e-4d13-8adf-60719afc2a10 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:26:50.029678 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70e0663-f76e-4d13-8adf-60719afc2a10 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:26:50.034247 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c91d0f8-749a-4ed2-86c2-26aa25fca976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:26:50.034898 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c91d0f8-749a-4ed2-86c2-26aa25fca976 admin admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:26:50.035182 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c91d0f8-749a-4ed2-86c2-26aa25fca976 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:26:50.035476 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c91d0f8-749a-4ed2-86c2-26aa25fca976 admin admin] PATH_INFO: `/v3/projects/3173698a626e49379cc39d87f0988d86/users/271c6b0f55da4867b0ee3d5d05f83257/roles/ef689f96442945e8a7bc8cbfee13d909` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:26:50.052103 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c91d0f8-749a-4ed2-86c2-26aa25fca976 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3173698a626e49379cc39d87f0988d86, user_id=271c6b0f55da4867b0ee3d5d05f83257, role_id=ef689f96442945e8a7bc8cbfee13d909)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:26:50.054822 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c91d0f8-749a-4ed2-86c2-26aa25fca976 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:26:50.111901 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1064/2134] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:26:50 2022] PUT /identity/v3/projects/3173698a626e49379cc39d87f0988d86/users/271c6b0f55da4867b0ee3d5d05f83257/roles/ef689f96442945e8a7bc8cbfee13d909 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:26:50.127777 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ba169-7a89-44c4-b7fe-52710b68a69f None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:26:50.128082 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ba169-7a89-44c4-b7fe-52710b68a69f None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:26:50.128391 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ba169-7a89-44c4-b7fe-52710b68a69f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:26:50.156701 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-a37ba169-7a89-44c4-b7fe-52710b68a69f None None] MFA Rules not processed for user `271c6b0f55da4867b0ee3d5d05f83257`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:26:50.157900 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-a37ba169-7a89-44c4-b7fe-52710b68a69f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:26:50.223191 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1071/2135] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:26:50 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:26:50.234029 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c91d0f8-749a-4ed2-86c2-26aa25fca976 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:26:50.235772 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-9c91d0f8-749a-4ed2-86c2-26aa25fca976 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:26:50.293880 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c91d0f8-749a-4ed2-86c2-26aa25fca976 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:26:50.297706 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14181ec9-5eeb-458f-a9af-326277cf9df8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:26:50.298450 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14181ec9-5eeb-458f-a9af-326277cf9df8 service cinder] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:26:50.298570 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14181ec9-5eeb-458f-a9af-326277cf9df8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:26:50.298836 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14181ec9-5eeb-458f-a9af-326277cf9df8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:26:50.308682 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14181ec9-5eeb-458f-a9af-326277cf9df8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:26:50.310124 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14181ec9-5eeb-458f-a9af-326277cf9df8 service cinder] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:26:50.318961 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1065/2136] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:26:50 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:26:56.742612 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4c2e86-7458-4be0-8bcf-7f0d1d04f2d4 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:26:56.743100 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4c2e86-7458-4be0-8bcf-7f0d1d04f2d4 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:26:56.743195 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4c2e86-7458-4be0-8bcf-7f0d1d04f2d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:26:56.772686 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-dd4c2e86-7458-4be0-8bcf-7f0d1d04f2d4 None None] MFA Rules not processed for user `d030d18b595144f19d6ad01d4cd5d52b`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:26:56.773895 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-dd4c2e86-7458-4be0-8bcf-7f0d1d04f2d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:26:56.832992 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1072/2137] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Oct 12 00:26:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:26:56.984355 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14181ec9-5eeb-458f-a9af-326277cf9df8 service cinder] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:26:56.985851 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-14181ec9-5eeb-458f-a9af-326277cf9df8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:26:57.058336 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14181ec9-5eeb-458f-a9af-326277cf9df8 service cinder] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:26:57.063328 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a40388-e158-4290-8430-12185ae948ac service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc1bcffc78844dcbb1a9b7c80285249b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:26:57.063987 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a40388-e158-4290-8430-12185ae948ac service swift] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:26:57.064271 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a40388-e158-4290-8430-12185ae948ac service swift] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:26:57.064546 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a40388-e158-4290-8430-12185ae948ac service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:26:57.072814 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a40388-e158-4290-8430-12185ae948ac service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:26:57.076928 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a40388-e158-4290-8430-12185ae948ac service swift] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:26:57.083382 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1066/2138] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Oct 12 00:26:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 12 00:27:06.303207 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4c2e86-7458-4be0-8bcf-7f0d1d04f2d4 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:06.305353 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-dd4c2e86-7458-4be0-8bcf-7f0d1d04f2d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:06.484321 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:27:06.484321 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4c2e86-7458-4be0-8bcf-7f0d1d04f2d4 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:06.484321 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca587712-86f3-479b-9f7e-791578394470 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:06.484321 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca587712-86f3-479b-9f7e-791578394470 service neutron] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:06.484674 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca587712-86f3-479b-9f7e-791578394470 service neutron] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:06.484674 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca587712-86f3-479b-9f7e-791578394470 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:06.484674 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-ca587712-86f3-479b-9f7e-791578394470 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:06.484674 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca587712-86f3-479b-9f7e-791578394470 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:06.484674 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca587712-86f3-479b-9f7e-791578394470 service neutron] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:06.484674 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1073/2139] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:27:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 00:27:07.079389 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=3 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:27:10.715223 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a40388-e158-4290-8430-12185ae948ac service swift] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:10.717522 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-b9a40388-e158-4290-8430-12185ae948ac service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:10.796059 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a40388-e158-4290-8430-12185ae948ac service swift] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:10.801155 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249c747b-60ec-4dbf-90db-558d3916cdff service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e528da752ed4c78bc747215909f6569', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:10.803065 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c747b-60ec-4dbf-90db-558d3916cdff service placement] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:10.803346 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c747b-60ec-4dbf-90db-558d3916cdff service placement] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:10.803681 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c747b-60ec-4dbf-90db-558d3916cdff service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:10.805439 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-249c747b-60ec-4dbf-90db-558d3916cdff service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:10.817429 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249c747b-60ec-4dbf-90db-558d3916cdff service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:10.819043 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249c747b-60ec-4dbf-90db-558d3916cdff service placement] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:10.829673 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1067/2140] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:27:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:27:10.845074 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca587712-86f3-479b-9f7e-791578394470 service neutron] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:10.847971 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-ca587712-86f3-479b-9f7e-791578394470 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:10.867392 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca587712-86f3-479b-9f7e-791578394470 service neutron] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:10.873149 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da38ebc-e0dd-4623-b094-576654a6ed87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:10.874043 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da38ebc-e0dd-4623-b094-576654a6ed87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:10.874508 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da38ebc-e0dd-4623-b094-576654a6ed87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:10.874935 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da38ebc-e0dd-4623-b094-576654a6ed87 admin admin] PATH_INFO: `/v3/users/bca53aca7b134df1949deb10858f14d0` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:10.877358 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da38ebc-e0dd-4623-b094-576654a6ed87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bca53aca7b134df1949deb10858f14d0)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:10.880611 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da38ebc-e0dd-4623-b094-576654a6ed87 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:11.100951 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-5da38ebc-e0dd-4623-b094-576654a6ed87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bca53aca7b134df1949deb10858f14d0'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:11.103200 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-5da38ebc-e0dd-4623-b094-576654a6ed87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bca53aca7b134df1949deb10858f14d0'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:11.114092 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-5da38ebc-e0dd-4623-b094-576654a6ed87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bca53aca7b134df1949deb10858f14d0'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:11.185200 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-5da38ebc-e0dd-4623-b094-576654a6ed87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bca53aca7b134df1949deb10858f14d0'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:11.185858 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-5da38ebc-e0dd-4623-b094-576654a6ed87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bca53aca7b134df1949deb10858f14d0'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:11.199751 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1074/2141] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:27:10 2022] DELETE /identity/v3/users/bca53aca7b134df1949deb10858f14d0 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:27:11.450321 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249c747b-60ec-4dbf-90db-558d3916cdff service placement] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:11.463759 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249c747b-60ec-4dbf-90db-558d3916cdff service placement] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:11.528371 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6166079-b51e-4712-abcd-56a72d8a1eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:11.529011 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6166079-b51e-4712-abcd-56a72d8a1eca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:11.529293 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6166079-b51e-4712-abcd-56a72d8a1eca admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:11.529568 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6166079-b51e-4712-abcd-56a72d8a1eca admin admin] PATH_INFO: `/v3/projects/54567f4ea839415a94397e13f93d71cc` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:11.530708 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6166079-b51e-4712-abcd-56a72d8a1eca admin admin] RBAC: Authorizing `identity:delete_project(project_id=54567f4ea839415a94397e13f93d71cc)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:11.532686 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6166079-b51e-4712-abcd-56a72d8a1eca admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:11.705851 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-e6166079-b51e-4712-abcd-56a72d8a1eca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54567f4ea839415a94397e13f93d71cc', 'request_id': 'req-e6166079-b51e-4712-abcd-56a72d8a1eca'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:11.720341 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-e6166079-b51e-4712-abcd-56a72d8a1eca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54567f4ea839415a94397e13f93d71cc', 'request_id': 'req-e6166079-b51e-4712-abcd-56a72d8a1eca'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:11.787014 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-e6166079-b51e-4712-abcd-56a72d8a1eca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54567f4ea839415a94397e13f93d71cc', 'request_id': 'req-e6166079-b51e-4712-abcd-56a72d8a1eca'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:11.799339 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-e6166079-b51e-4712-abcd-56a72d8a1eca admin admin] The token cache is being invalidate because project 54567f4ea839415a94397e13f93d71cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81746) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:27:11.799584 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-e6166079-b51e-4712-abcd-56a72d8a1eca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:11.801373 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-e6166079-b51e-4712-abcd-56a72d8a1eca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:11.803470 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1068/2142] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:27:11 2022] DELETE /identity/v3/projects/54567f4ea839415a94397e13f93d71cc => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:27:11.808562 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0516de-c142-435e-898e-1b7154e55fb0 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:11.808848 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0516de-c142-435e-898e-1b7154e55fb0 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:11.810571 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0516de-c142-435e-898e-1b7154e55fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:11.829846 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-ee0516de-c142-435e-898e-1b7154e55fb0 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:27:11.831037 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-ee0516de-c142-435e-898e-1b7154e55fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:11.883360 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:27:11.896338 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1075/2143] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:27:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:27:11.905606 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d34881e-b03f-4108-8ef1-9cdf7b58b650 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:11.905969 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d34881e-b03f-4108-8ef1-9cdf7b58b650 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:11.906310 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d34881e-b03f-4108-8ef1-9cdf7b58b650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:11.927358 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-8d34881e-b03f-4108-8ef1-9cdf7b58b650 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:27:11.928606 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-8d34881e-b03f-4108-8ef1-9cdf7b58b650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:11.937659 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1069/2144] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:27:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:27:11.941699 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0516de-c142-435e-898e-1b7154e55fb0 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:11.953498 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0516de-c142-435e-898e-1b7154e55fb0 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:11.958955 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f163048-720e-472a-9093-3799874d5542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:11.959755 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d34881e-b03f-4108-8ef1-9cdf7b58b650 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:11.960129 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f163048-720e-472a-9093-3799874d5542 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:11.965493 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f163048-720e-472a-9093-3799874d5542 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:11.965493 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f163048-720e-472a-9093-3799874d5542 admin admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:11.965862 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-8d34881e-b03f-4108-8ef1-9cdf7b58b650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:11.965957 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f163048-720e-472a-9093-3799874d5542 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:27:11.965957 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f163048-720e-472a-9093-3799874d5542 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:11.965957 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f163048-720e-472a-9093-3799874d5542 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:11.974109 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d34881e-b03f-4108-8ef1-9cdf7b58b650 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:11.976862 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1076/2145] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Oct 12 00:27:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:27:11.979286 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dd4110-335c-4f2e-a2ac-d817b2b50b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:11.980238 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dd4110-335c-4f2e-a2ac-d817b2b50b70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:11.980727 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dd4110-335c-4f2e-a2ac-d817b2b50b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:11.981191 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dd4110-335c-4f2e-a2ac-d817b2b50b70 admin admin] PATH_INFO: `/v3/users/271c6b0f55da4867b0ee3d5d05f83257` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:11.981301 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f163048-720e-472a-9093-3799874d5542 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:11.982662 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dd4110-335c-4f2e-a2ac-d817b2b50b70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=271c6b0f55da4867b0ee3d5d05f83257)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:11.984803 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dd4110-335c-4f2e-a2ac-d817b2b50b70 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:11.993257 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f163048-720e-472a-9093-3799874d5542 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:11.994058 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:27:11.998285 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d846bfed-3aa0-4914-9738-fc22d1c10df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:11.999926 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d846bfed-3aa0-4914-9738-fc22d1c10df1 admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:11.999926 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d846bfed-3aa0-4914-9738-fc22d1c10df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:11.999926 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d846bfed-3aa0-4914-9738-fc22d1c10df1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:12.000343 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d846bfed-3aa0-4914-9738-fc22d1c10df1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:12.002050 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d846bfed-3aa0-4914-9738-fc22d1c10df1 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:12.113909 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1077/2146] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:27:11 2022] POST /identity/v3/projects => generated 383 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 00:27:12.117826 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d846bfed-3aa0-4914-9738-fc22d1c10df1 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:12.179880 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-41dd4110-335c-4f2e-a2ac-d817b2b50b70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '271c6b0f55da4867b0ee3d5d05f83257'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:12.180699 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-41dd4110-335c-4f2e-a2ac-d817b2b50b70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '271c6b0f55da4867b0ee3d5d05f83257'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:12.204015 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-41dd4110-335c-4f2e-a2ac-d817b2b50b70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '271c6b0f55da4867b0ee3d5d05f83257'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:12.206233 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d846bfed-3aa0-4914-9738-fc22d1c10df1 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:12.211797 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f49098-0a39-40c5-8992-9ce7394130ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:12.212554 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f49098-0a39-40c5-8992-9ce7394130ac admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:12.212895 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f49098-0a39-40c5-8992-9ce7394130ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:12.213240 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f49098-0a39-40c5-8992-9ce7394130ac admin admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:12.214011 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f49098-0a39-40c5-8992-9ce7394130ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:12.216047 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f49098-0a39-40c5-8992-9ce7394130ac admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:12.266621 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-41dd4110-335c-4f2e-a2ac-d817b2b50b70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '271c6b0f55da4867b0ee3d5d05f83257'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:12.267797 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-41dd4110-335c-4f2e-a2ac-d817b2b50b70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '271c6b0f55da4867b0ee3d5d05f83257'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:12.283308 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1070/2147] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:27:11 2022] DELETE /identity/v3/users/271c6b0f55da4867b0ee3d5d05f83257 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:27:12.288330 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e200b4ed-7db7-495c-8195-8c69cb291422 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:12.288773 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e200b4ed-7db7-495c-8195-8c69cb291422 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:12.289357 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e200b4ed-7db7-495c-8195-8c69cb291422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:12.309609 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-e200b4ed-7db7-495c-8195-8c69cb291422 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:27:12.315015 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-e200b4ed-7db7-495c-8195-8c69cb291422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:12.315157 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1078/2148] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:27:12 2022] POST /identity/v3/users => generated 292 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 00:27:12.319090 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f49098-0a39-40c5-8992-9ce7394130ac admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:12.320924 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-f5f49098-0a39-40c5-8992-9ce7394130ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:12.335828 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f49098-0a39-40c5-8992-9ce7394130ac admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:12.381382 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1071/2149] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:27:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:27:12.391229 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e200b4ed-7db7-495c-8195-8c69cb291422 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:12.393072 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-e200b4ed-7db7-495c-8195-8c69cb291422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:12.410751 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67ca642-3362-4f08-9a39-586d3d763c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:12.411395 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ca642-3362-4f08-9a39-586d3d763c7c admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:12.411680 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ca642-3362-4f08-9a39-586d3d763c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:12.411952 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ca642-3362-4f08-9a39-586d3d763c7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:12.412608 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67ca642-3362-4f08-9a39-586d3d763c7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:12.415337 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67ca642-3362-4f08-9a39-586d3d763c7c admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:12.426785 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1079/2150] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:27:12 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:27:12.430468 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67ca642-3362-4f08-9a39-586d3d763c7c admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:12.441384 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67ca642-3362-4f08-9a39-586d3d763c7c admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:12.445804 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475ba10c-9b93-45b1-8214-755da00aa679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:12.446412 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475ba10c-9b93-45b1-8214-755da00aa679 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:12.446753 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475ba10c-9b93-45b1-8214-755da00aa679 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:12.446950 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475ba10c-9b93-45b1-8214-755da00aa679 admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:12.447610 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475ba10c-9b93-45b1-8214-755da00aa679 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:12.449201 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475ba10c-9b93-45b1-8214-755da00aa679 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:12.455466 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e200b4ed-7db7-495c-8195-8c69cb291422 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:12.460051 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b41012-3d68-4336-992c-ec71446dc32a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:12.460747 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b41012-3d68-4336-992c-ec71446dc32a service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:12.460879 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1080/2151] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:27:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 00:27:12.461148 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b41012-3d68-4336-992c-ec71446dc32a service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:12.461450 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b41012-3d68-4336-992c-ec71446dc32a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:12.465179 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475ba10c-9b93-45b1-8214-755da00aa679 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:12.471143 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b41012-3d68-4336-992c-ec71446dc32a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:12.472806 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b41012-3d68-4336-992c-ec71446dc32a service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:12.475038 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475ba10c-9b93-45b1-8214-755da00aa679 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:12.480967 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f269dd54-f618-478f-96aa-0c77e61785aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:12.481736 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f269dd54-f618-478f-96aa-0c77e61785aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:12.482015 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f269dd54-f618-478f-96aa-0c77e61785aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:12.482288 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f269dd54-f618-478f-96aa-0c77e61785aa admin admin] PATH_INFO: `/v3/projects/85a220a727d24cb48b8902f6787479a5/users/d81c6833a09244229484527eb694fca8/roles/ef689f96442945e8a7bc8cbfee13d909` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:12.483064 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1072/2152] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:27:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:27:12.498840 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f269dd54-f618-478f-96aa-0c77e61785aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=85a220a727d24cb48b8902f6787479a5, user_id=d81c6833a09244229484527eb694fca8, role_id=ef689f96442945e8a7bc8cbfee13d909)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:12.501736 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f269dd54-f618-478f-96aa-0c77e61785aa admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:12.564278 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1081/2153] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:27:12 2022] PUT /identity/v3/projects/85a220a727d24cb48b8902f6787479a5/users/d81c6833a09244229484527eb694fca8/roles/ef689f96442945e8a7bc8cbfee13d909 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:27:12.575990 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae18266-8552-442e-b65e-842422684e6d None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:12.576281 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae18266-8552-442e-b65e-842422684e6d None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:12.576558 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae18266-8552-442e-b65e-842422684e6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:12.610333 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-aae18266-8552-442e-b65e-842422684e6d None None] MFA Rules not processed for user `d81c6833a09244229484527eb694fca8`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:27:12.611598 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-aae18266-8552-442e-b65e-842422684e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:12.635853 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:27:12.687485 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1073/2154] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:27:12 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:27:12.702129 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f269dd54-f618-478f-96aa-0c77e61785aa admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:12.703989 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-f269dd54-f618-478f-96aa-0c77e61785aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:12.721652 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:27:12.805840 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f269dd54-f618-478f-96aa-0c77e61785aa admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:12.826891 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2442ce95-5b0d-4933-b523-707fc384c5ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a515b75aa3944912ad2364ffd7924f40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:12.829257 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2442ce95-5b0d-4933-b523-707fc384c5ab service nova] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:12.829539 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2442ce95-5b0d-4933-b523-707fc384c5ab service nova] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:12.829941 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2442ce95-5b0d-4933-b523-707fc384c5ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:12.839160 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2442ce95-5b0d-4933-b523-707fc384c5ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:12.841109 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2442ce95-5b0d-4933-b523-707fc384c5ab service nova] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:12.863967 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1082/2155] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:27:12 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:27:12.902178 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae18266-8552-442e-b65e-842422684e6d None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:12.903657 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-aae18266-8552-442e-b65e-842422684e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:12.977662 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae18266-8552-442e-b65e-842422684e6d None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:12.984077 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab2fcb5-ba48-4a3a-a624-aada4cc4f8b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:12.984755 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab2fcb5-ba48-4a3a-a624-aada4cc4f8b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:12.985035 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab2fcb5-ba48-4a3a-a624-aada4cc4f8b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:12.987459 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab2fcb5-ba48-4a3a-a624-aada4cc4f8b6 admin admin] PATH_INFO: `/v3/projects/3173698a626e49379cc39d87f0988d86` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:12.987459 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab2fcb5-ba48-4a3a-a624-aada4cc4f8b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3173698a626e49379cc39d87f0988d86)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:12.989127 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab2fcb5-ba48-4a3a-a624-aada4cc4f8b6 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:13.089430 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-cab2fcb5-ba48-4a3a-a624-aada4cc4f8b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3173698a626e49379cc39d87f0988d86', 'request_id': 'req-cab2fcb5-ba48-4a3a-a624-aada4cc4f8b6'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:13.097543 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-cab2fcb5-ba48-4a3a-a624-aada4cc4f8b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3173698a626e49379cc39d87f0988d86', 'request_id': 'req-cab2fcb5-ba48-4a3a-a624-aada4cc4f8b6'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:13.168950 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-cab2fcb5-ba48-4a3a-a624-aada4cc4f8b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3173698a626e49379cc39d87f0988d86', 'request_id': 'req-cab2fcb5-ba48-4a3a-a624-aada4cc4f8b6'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:13.180447 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-cab2fcb5-ba48-4a3a-a624-aada4cc4f8b6 admin admin] The token cache is being invalidate because project 3173698a626e49379cc39d87f0988d86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81746) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:27:13.180740 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-cab2fcb5-ba48-4a3a-a624-aada4cc4f8b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:13.181380 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-cab2fcb5-ba48-4a3a-a624-aada4cc4f8b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:13.182919 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1074/2156] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:27:12 2022] DELETE /identity/v3/projects/3173698a626e49379cc39d87f0988d86 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:27:13.189090 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d6379-e144-4c90-9d48-8ff2e1166bf1 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:13.194872 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d6379-e144-4c90-9d48-8ff2e1166bf1 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:13.194872 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d6379-e144-4c90-9d48-8ff2e1166bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:13.209253 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-5c8d6379-e144-4c90-9d48-8ff2e1166bf1 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:27:13.210474 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-5c8d6379-e144-4c90-9d48-8ff2e1166bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:13.274156 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1083/2157] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:27:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:27:13.343910 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5835f97-b88b-4d37-bdfe-c979d2c6f33b None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:13.344696 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5835f97-b88b-4d37-bdfe-c979d2c6f33b None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:13.345212 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5835f97-b88b-4d37-bdfe-c979d2c6f33b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:13.397161 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-d5835f97-b88b-4d37-bdfe-c979d2c6f33b None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:27:13.399302 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-d5835f97-b88b-4d37-bdfe-c979d2c6f33b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:13.410012 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1075/2158] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:27:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:27:13.414089 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8d6379-e144-4c90-9d48-8ff2e1166bf1 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:13.427942 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8d6379-e144-4c90-9d48-8ff2e1166bf1 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:13.432870 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275eb77f-145c-4cf2-8a92-7aec936223c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:13.433502 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275eb77f-145c-4cf2-8a92-7aec936223c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:13.433777 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275eb77f-145c-4cf2-8a92-7aec936223c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:13.434046 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275eb77f-145c-4cf2-8a92-7aec936223c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:13.434708 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275eb77f-145c-4cf2-8a92-7aec936223c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:27:13.435037 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275eb77f-145c-4cf2-8a92-7aec936223c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:13.436903 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275eb77f-145c-4cf2-8a92-7aec936223c1 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:13.456533 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1084/2159] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Oct 12 00:27:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:27:13.457848 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5835f97-b88b-4d37-bdfe-c979d2c6f33b None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:13.468195 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5835f97-b88b-4d37-bdfe-c979d2c6f33b None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:13.474622 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b53f975-a86a-47ac-98f5-f697357c9c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:13.476731 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b53f975-a86a-47ac-98f5-f697357c9c09 admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:13.476731 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b53f975-a86a-47ac-98f5-f697357c9c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:13.476731 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b53f975-a86a-47ac-98f5-f697357c9c09 admin admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:13.476731 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b53f975-a86a-47ac-98f5-f697357c9c09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:13.478600 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b53f975-a86a-47ac-98f5-f697357c9c09 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:13.604269 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1076/2160] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:27:13 2022] POST /identity/v3/projects => generated 371 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:27:13.608440 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275eb77f-145c-4cf2-8a92-7aec936223c1 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:13.619350 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275eb77f-145c-4cf2-8a92-7aec936223c1 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:13.727877 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80ad480-70fe-4bed-ad00-c749a086300d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:13.728490 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ad480-70fe-4bed-ad00-c749a086300d admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:13.728765 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ad480-70fe-4bed-ad00-c749a086300d admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:13.729032 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ad480-70fe-4bed-ad00-c749a086300d admin admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:13.729656 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80ad480-70fe-4bed-ad00-c749a086300d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:13.731726 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80ad480-70fe-4bed-ad00-c749a086300d admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:13.889636 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1085/2161] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:27:13 2022] POST /identity/v3/users => generated 286 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:27:13.894416 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b53f975-a86a-47ac-98f5-f697357c9c09 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:13.904967 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b53f975-a86a-47ac-98f5-f697357c9c09 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:13.909676 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b47f822-81ea-483d-bbbe-abebb3ca6ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:13.910314 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b47f822-81ea-483d-bbbe-abebb3ca6ef2 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:13.910592 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b47f822-81ea-483d-bbbe-abebb3ca6ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:13.910867 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b47f822-81ea-483d-bbbe-abebb3ca6ef2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:13.911545 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b47f822-81ea-483d-bbbe-abebb3ca6ef2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:13.913142 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b47f822-81ea-483d-bbbe-abebb3ca6ef2 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:13.926592 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1077/2162] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:27:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:27:13.928104 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80ad480-70fe-4bed-ad00-c749a086300d admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:13.938817 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80ad480-70fe-4bed-ad00-c749a086300d admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:13.944244 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869be9f6-125b-49ba-839f-f8345b81614b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:13.945148 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869be9f6-125b-49ba-839f-f8345b81614b admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:13.945313 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869be9f6-125b-49ba-839f-f8345b81614b admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:13.945467 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869be9f6-125b-49ba-839f-f8345b81614b admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:13.946201 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869be9f6-125b-49ba-839f-f8345b81614b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:13.947939 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869be9f6-125b-49ba-839f-f8345b81614b admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:13.960147 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1086/2163] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:27:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:27:13.964399 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b47f822-81ea-483d-bbbe-abebb3ca6ef2 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:13.974586 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b47f822-81ea-483d-bbbe-abebb3ca6ef2 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:13.979318 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0e4ada-94b8-4ed5-9ad9-83f54076eb75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:13.980005 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e4ada-94b8-4ed5-9ad9-83f54076eb75 admin admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:13.980278 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e4ada-94b8-4ed5-9ad9-83f54076eb75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:13.980553 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e4ada-94b8-4ed5-9ad9-83f54076eb75 admin admin] PATH_INFO: `/v3/projects/654843d065264b9e9296824daf702110/users/da2a3fcb0287474cb93238923aa9c7c1/roles/ef689f96442945e8a7bc8cbfee13d909` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:13.984998 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:27:13.996699 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0e4ada-94b8-4ed5-9ad9-83f54076eb75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=654843d065264b9e9296824daf702110, user_id=da2a3fcb0287474cb93238923aa9c7c1, role_id=ef689f96442945e8a7bc8cbfee13d909)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:14.000551 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0e4ada-94b8-4ed5-9ad9-83f54076eb75 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:14.100327 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1078/2164] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:27:13 2022] PUT /identity/v3/projects/654843d065264b9e9296824daf702110/users/da2a3fcb0287474cb93238923aa9c7c1/roles/ef689f96442945e8a7bc8cbfee13d909 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:27:14.105385 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9201c24c-3f75-4dcb-b2bb-501f48bc2659 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:14.105664 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9201c24c-3f75-4dcb-b2bb-501f48bc2659 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:14.105932 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9201c24c-3f75-4dcb-b2bb-501f48bc2659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:14.113092 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:27:14.125551 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-9201c24c-3f75-4dcb-b2bb-501f48bc2659 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:27:14.126852 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-9201c24c-3f75-4dcb-b2bb-501f48bc2659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:14.197231 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1087/2165] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:27:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:27:14.209192 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0e4ada-94b8-4ed5-9ad9-83f54076eb75 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:14.210913 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-de0e4ada-94b8-4ed5-9ad9-83f54076eb75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:14.270350 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0e4ada-94b8-4ed5-9ad9-83f54076eb75 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:14.274330 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c66385-fd14-446c-8fa2-a6d13f2bd22f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:14.275020 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c66385-fd14-446c-8fa2-a6d13f2bd22f service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:14.275304 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c66385-fd14-446c-8fa2-a6d13f2bd22f service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:14.275663 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c66385-fd14-446c-8fa2-a6d13f2bd22f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:14.285184 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c66385-fd14-446c-8fa2-a6d13f2bd22f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:14.286621 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c66385-fd14-446c-8fa2-a6d13f2bd22f service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:14.295981 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1079/2166] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:27:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:27:18.853562 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9201c24c-3f75-4dcb-b2bb-501f48bc2659 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:18.870263 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9201c24c-3f75-4dcb-b2bb-501f48bc2659 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:18.875024 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8127fbcd-4b45-4804-962f-1931e860a860 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:18.875751 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8127fbcd-4b45-4804-962f-1931e860a860 service neutron] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:18.876135 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8127fbcd-4b45-4804-962f-1931e860a860 service neutron] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:18.876454 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8127fbcd-4b45-4804-962f-1931e860a860 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:18.878626 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-8127fbcd-4b45-4804-962f-1931e860a860 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:18.892079 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8127fbcd-4b45-4804-962f-1931e860a860 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:18.893480 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8127fbcd-4b45-4804-962f-1931e860a860 service neutron] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:18.904789 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1088/2167] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:27:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:27:19.439988 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a572d0cc-bdc0-45c4-b590-929c161df69a None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:19.440512 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a572d0cc-bdc0-45c4-b590-929c161df69a None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:19.440654 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a572d0cc-bdc0-45c4-b590-929c161df69a None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:19.442236 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1080/2168] 10.222.0.36 () {54 vars in 921 bytes} [Wed Oct 12 00:27:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:27:19.448568 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2a4f61-8ac7-4c03-ba98-744e26c8362d None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:19.449452 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2a4f61-8ac7-4c03-ba98-744e26c8362d None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:19.449789 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2a4f61-8ac7-4c03-ba98-744e26c8362d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:19.481131 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-8f2a4f61-8ac7-4c03-ba98-744e26c8362d None None] MFA Rules not processed for user `a515b75aa3944912ad2364ffd7924f40`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:27:19.482437 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-8f2a4f61-8ac7-4c03-ba98-744e26c8362d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:19.587639 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1089/2169] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Oct 12 00:27:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:27:19.599808 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a572d0cc-bdc0-45c4-b590-929c161df69a None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:19.604525 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-a572d0cc-bdc0-45c4-b590-929c161df69a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:19.704440 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a572d0cc-bdc0-45c4-b590-929c161df69a None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:19.710986 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3049156f-4caa-42c6-8101-6579e11e3193 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59b883abbb674b3c976bacb64194a66d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:19.711952 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3049156f-4caa-42c6-8101-6579e11e3193 service glance] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:19.712426 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3049156f-4caa-42c6-8101-6579e11e3193 service glance] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:19.712908 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3049156f-4caa-42c6-8101-6579e11e3193 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:19.732384 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3049156f-4caa-42c6-8101-6579e11e3193 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:19.733990 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3049156f-4caa-42c6-8101-6579e11e3193 service glance] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:19.747571 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1081/2170] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:27:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:27:20.061754 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d50d08-950b-404f-80d8-bb173b23fcf9 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:20.062636 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d50d08-950b-404f-80d8-bb173b23fcf9 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:20.063060 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d50d08-950b-404f-80d8-bb173b23fcf9 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:20.064639 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1090/2171] 10.222.0.36 () {54 vars in 921 bytes} [Wed Oct 12 00:27:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:27:20.069871 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42214b1b-84d9-46e9-8b81-3cda0522f900 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:20.070237 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42214b1b-84d9-46e9-8b81-3cda0522f900 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:20.070590 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42214b1b-84d9-46e9-8b81-3cda0522f900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:20.092700 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-42214b1b-84d9-46e9-8b81-3cda0522f900 None None] MFA Rules not processed for user `51215dab618141b4991e232724094140`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:27:20.094083 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-42214b1b-84d9-46e9-8b81-3cda0522f900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:20.110316 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1082/2172] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Oct 12 00:27:20 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:27:20.111986 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47166d7a-17d9-46e8-81d1-1736ba26ed0a None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:20.112264 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47166d7a-17d9-46e8-81d1-1736ba26ed0a None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:20.113152 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47166d7a-17d9-46e8-81d1-1736ba26ed0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:20.119033 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42214b1b-84d9-46e9-8b81-3cda0522f900 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:20.119033 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-42214b1b-84d9-46e9-8b81-3cda0522f900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:20.130618 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42214b1b-84d9-46e9-8b81-3cda0522f900 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:20.134832 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb11a76-1e34-41db-bb54-509fc739bea6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:20.135679 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb11a76-1e34-41db-bb54-509fc739bea6 service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:20.136115 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb11a76-1e34-41db-bb54-509fc739bea6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:20.136542 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb11a76-1e34-41db-bb54-509fc739bea6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:20.154898 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb11a76-1e34-41db-bb54-509fc739bea6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:20.156993 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb11a76-1e34-41db-bb54-509fc739bea6 service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:20.157976 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-47166d7a-17d9-46e8-81d1-1736ba26ed0a None None] MFA Rules not processed for user `da2a3fcb0287474cb93238923aa9c7c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:27:20.159967 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-47166d7a-17d9-46e8-81d1-1736ba26ed0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:20.176226 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1083/2173] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:27:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:27:20.305401 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1091/2174] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:27:20 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:27:20.308605 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb11a76-1e34-41db-bb54-509fc739bea6 service neutron] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:20.320276 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740af78b-63a1-4e70-a97c-795b2e4bde41 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:20.320555 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740af78b-63a1-4e70-a97c-795b2e4bde41 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:20.321147 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740af78b-63a1-4e70-a97c-795b2e4bde41 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:20.325722 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb11a76-1e34-41db-bb54-509fc739bea6 service neutron] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:20.326755 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1092/2175] 10.222.0.36 () {54 vars in 921 bytes} [Wed Oct 12 00:27:20 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 00:27:20.330347 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92efdec4-a57f-4b69-8c77-adb0d127117b None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:20.330625 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92efdec4-a57f-4b69-8c77-adb0d127117b None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:20.331973 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92efdec4-a57f-4b69-8c77-adb0d127117b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:20.338126 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc0e092-c082-4e96-b28c-c682295459f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:20.339514 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc0e092-c082-4e96-b28c-c682295459f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:20.339795 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc0e092-c082-4e96-b28c-c682295459f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:20.345443 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc0e092-c082-4e96-b28c-c682295459f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:20.345443 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc0e092-c082-4e96-b28c-c682295459f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:20.345873 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc0e092-c082-4e96-b28c-c682295459f0 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:20.373014 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-92efdec4-a57f-4b69-8c77-adb0d127117b None None] MFA Rules not processed for user `a515b75aa3944912ad2364ffd7924f40`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:27:20.374702 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-92efdec4-a57f-4b69-8c77-adb0d127117b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:20.386235 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1093/2176] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Oct 12 00:27:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:27:20.398100 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92efdec4-a57f-4b69-8c77-adb0d127117b None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:20.404675 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:27:20.408628 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92efdec4-a57f-4b69-8c77-adb0d127117b None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:20.413236 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc93c1e-b80d-43d1-920d-dfb4c61a7120 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:20.413844 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc93c1e-b80d-43d1-920d-dfb4c61a7120 service neutron] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:20.414121 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc93c1e-b80d-43d1-920d-dfb4c61a7120 service neutron] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:20.414417 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc93c1e-b80d-43d1-920d-dfb4c61a7120 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:20.424954 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc93c1e-b80d-43d1-920d-dfb4c61a7120 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:20.426418 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc93c1e-b80d-43d1-920d-dfb4c61a7120 service neutron] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:20.437258 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1094/2177] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:27:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 00:27:20.445616 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1084/2178] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:27:20 2022] POST /identity/v3/projects => generated 373 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 00:27:20.449571 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc93c1e-b80d-43d1-920d-dfb4c61a7120 service neutron] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:20.460500 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc93c1e-b80d-43d1-920d-dfb4c61a7120 service neutron] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:20.518530 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffabdfa-7d74-4643-890a-0b485d5c65d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:20.519650 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffabdfa-7d74-4643-890a-0b485d5c65d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:20.519939 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffabdfa-7d74-4643-890a-0b485d5c65d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:20.520213 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffabdfa-7d74-4643-890a-0b485d5c65d7 admin admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:20.520843 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffabdfa-7d74-4643-890a-0b485d5c65d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:20.522787 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffabdfa-7d74-4643-890a-0b485d5c65d7 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:20.637710 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1095/2179] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:27:20 2022] POST /identity/v3/users => generated 293 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:27:20.637898 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc0e092-c082-4e96-b28c-c682295459f0 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:20.653781 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc0e092-c082-4e96-b28c-c682295459f0 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:20.658599 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9789a85b-f80d-46f8-8c57-cfc8ee28dfd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:20.659210 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9789a85b-f80d-46f8-8c57-cfc8ee28dfd2 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:20.659736 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9789a85b-f80d-46f8-8c57-cfc8ee28dfd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:20.660164 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9789a85b-f80d-46f8-8c57-cfc8ee28dfd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:20.660809 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9789a85b-f80d-46f8-8c57-cfc8ee28dfd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:20.662449 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9789a85b-f80d-46f8-8c57-cfc8ee28dfd2 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:20.673226 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1085/2180] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:27:20 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:27:20.677193 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffabdfa-7d74-4643-890a-0b485d5c65d7 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:20.687879 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffabdfa-7d74-4643-890a-0b485d5c65d7 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:20.693194 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f35ad1-fa46-4d40-921e-be423ff32bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:20.693894 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f35ad1-fa46-4d40-921e-be423ff32bdf admin admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:20.695736 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f35ad1-fa46-4d40-921e-be423ff32bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:20.696040 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f35ad1-fa46-4d40-921e-be423ff32bdf admin admin] PATH_INFO: `/v3/projects/406331653ce1431896c1d2ad89461310/users/269c39f34a014bcc87a1d47d2110105a/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:20.712723 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f35ad1-fa46-4d40-921e-be423ff32bdf admin admin] RBAC: Authorizing `identity:create_grant(project_id=406331653ce1431896c1d2ad89461310, user_id=269c39f34a014bcc87a1d47d2110105a, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:20.716555 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f35ad1-fa46-4d40-921e-be423ff32bdf admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:20.821554 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1096/2181] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:27:20 2022] PUT /identity/v3/projects/406331653ce1431896c1d2ad89461310/users/269c39f34a014bcc87a1d47d2110105a/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:27:20.825592 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9789a85b-f80d-46f8-8c57-cfc8ee28dfd2 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:20.836248 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9789a85b-f80d-46f8-8c57-cfc8ee28dfd2 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:20.870565 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:27:20.896181 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb06dc2-c55b-4b88-b9a0-d77b7a301103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:20.896815 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb06dc2-c55b-4b88-b9a0-d77b7a301103 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:20.897168 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb06dc2-c55b-4b88-b9a0-d77b7a301103 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:20.897440 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb06dc2-c55b-4b88-b9a0-d77b7a301103 admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:20.898117 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb06dc2-c55b-4b88-b9a0-d77b7a301103 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:20.899809 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb06dc2-c55b-4b88-b9a0-d77b7a301103 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:20.910166 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1086/2182] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:27:20 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:27:20.914307 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f35ad1-fa46-4d40-921e-be423ff32bdf admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:20.924998 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f35ad1-fa46-4d40-921e-be423ff32bdf admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:20.929762 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebd353b-a1e1-427e-863e-fa0f78c1e905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:20.930360 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebd353b-a1e1-427e-863e-fa0f78c1e905 admin admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:20.930637 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebd353b-a1e1-427e-863e-fa0f78c1e905 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:20.930912 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebd353b-a1e1-427e-863e-fa0f78c1e905 admin admin] PATH_INFO: `/v3/projects/406331653ce1431896c1d2ad89461310/users/269c39f34a014bcc87a1d47d2110105a/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:20.933012 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebd353b-a1e1-427e-863e-fa0f78c1e905 admin admin] RBAC: Authorizing `identity:create_grant(project_id=406331653ce1431896c1d2ad89461310, user_id=269c39f34a014bcc87a1d47d2110105a, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:20.935693 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebd353b-a1e1-427e-863e-fa0f78c1e905 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:20.947423 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1097/2183] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:27:20 2022] PUT /identity/v3/projects/406331653ce1431896c1d2ad89461310/users/269c39f34a014bcc87a1d47d2110105a/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 00:27:21.512224 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb06dc2-c55b-4b88-b9a0-d77b7a301103 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:21.512992 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-6fb06dc2-c55b-4b88-b9a0-d77b7a301103 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:21.526655 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb06dc2-c55b-4b88-b9a0-d77b7a301103 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:21.621102 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a612bf-4864-4a09-b724-5e48346f88ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:21.622148 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a612bf-4864-4a09-b724-5e48346f88ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:21.622302 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a612bf-4864-4a09-b724-5e48346f88ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:21.622742 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a612bf-4864-4a09-b724-5e48346f88ae admin admin] PATH_INFO: `/v3/users/d81c6833a09244229484527eb694fca8` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:21.627614 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a612bf-4864-4a09-b724-5e48346f88ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=d81c6833a09244229484527eb694fca8)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:21.632918 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a612bf-4864-4a09-b724-5e48346f88ae admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:21.890851 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-47a612bf-4864-4a09-b724-5e48346f88ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd81c6833a09244229484527eb694fca8'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:21.891563 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-47a612bf-4864-4a09-b724-5e48346f88ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd81c6833a09244229484527eb694fca8'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:21.930499 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-47a612bf-4864-4a09-b724-5e48346f88ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd81c6833a09244229484527eb694fca8'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:22.025111 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-47a612bf-4864-4a09-b724-5e48346f88ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd81c6833a09244229484527eb694fca8'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:22.025755 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-47a612bf-4864-4a09-b724-5e48346f88ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd81c6833a09244229484527eb694fca8'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:22.041087 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1087/2184] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:27:21 2022] DELETE /identity/v3/users/d81c6833a09244229484527eb694fca8 => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:27:22.045584 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f26ade-1e98-4adb-9b6a-7e4c31b7ce57 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:22.046134 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f26ade-1e98-4adb-9b6a-7e4c31b7ce57 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:22.046134 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f26ade-1e98-4adb-9b6a-7e4c31b7ce57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:22.066524 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-b4f26ade-1e98-4adb-9b6a-7e4c31b7ce57 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:27:22.067714 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-b4f26ade-1e98-4adb-9b6a-7e4c31b7ce57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:22.129496 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1098/2185] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:27:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:27:22.140621 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a612bf-4864-4a09-b724-5e48346f88ae admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:22.142757 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-47a612bf-4864-4a09-b724-5e48346f88ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:22.215020 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a612bf-4864-4a09-b724-5e48346f88ae admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:22.218863 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2a8589-4554-47be-87e0-3f83885c789e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:22.219823 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2a8589-4554-47be-87e0-3f83885c789e service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:22.220015 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2a8589-4554-47be-87e0-3f83885c789e service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:22.220255 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2a8589-4554-47be-87e0-3f83885c789e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:22.228017 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:27:22.230664 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2a8589-4554-47be-87e0-3f83885c789e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:22.232155 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2a8589-4554-47be-87e0-3f83885c789e service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:22.241866 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1088/2186] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:27:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:27:22.795345 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f26ade-1e98-4adb-9b6a-7e4c31b7ce57 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:22.796924 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-b4f26ade-1e98-4adb-9b6a-7e4c31b7ce57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:22.817703 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f26ade-1e98-4adb-9b6a-7e4c31b7ce57 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:22.833045 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0ddb31-4d58-478d-83a5-8b28fef38a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:22.833895 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ddb31-4d58-478d-83a5-8b28fef38a2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:22.835208 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ddb31-4d58-478d-83a5-8b28fef38a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:22.835208 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ddb31-4d58-478d-83a5-8b28fef38a2b admin admin] PATH_INFO: `/v3/projects/85a220a727d24cb48b8902f6787479a5` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:22.835825 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0ddb31-4d58-478d-83a5-8b28fef38a2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=85a220a727d24cb48b8902f6787479a5)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:22.837824 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0ddb31-4d58-478d-83a5-8b28fef38a2b admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:22.886652 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2a8589-4554-47be-87e0-3f83885c789e service neutron] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:22.888370 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-cd2a8589-4554-47be-87e0-3f83885c789e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:22.901780 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2a8589-4554-47be-87e0-3f83885c789e service neutron] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:22.906424 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2650b60-a66b-4869-b10c-2a3cb0d9f1f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:22.907242 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2650b60-a66b-4869-b10c-2a3cb0d9f1f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:22.907906 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2650b60-a66b-4869-b10c-2a3cb0d9f1f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:22.908326 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2650b60-a66b-4869-b10c-2a3cb0d9f1f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:22.910923 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-e2650b60-a66b-4869-b10c-2a3cb0d9f1f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:22.927076 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2650b60-a66b-4869-b10c-2a3cb0d9f1f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:22.928108 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2650b60-a66b-4869-b10c-2a3cb0d9f1f0 service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:22.941895 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1089/2187] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:27:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 00:27:22.986822 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:27:23.013416 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-9c0ddb31-4d58-478d-83a5-8b28fef38a2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85a220a727d24cb48b8902f6787479a5', 'request_id': 'req-9c0ddb31-4d58-478d-83a5-8b28fef38a2b'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:23.032135 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-9c0ddb31-4d58-478d-83a5-8b28fef38a2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85a220a727d24cb48b8902f6787479a5', 'request_id': 'req-9c0ddb31-4d58-478d-83a5-8b28fef38a2b'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:23.117734 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-9c0ddb31-4d58-478d-83a5-8b28fef38a2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85a220a727d24cb48b8902f6787479a5', 'request_id': 'req-9c0ddb31-4d58-478d-83a5-8b28fef38a2b'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:23.137041 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-9c0ddb31-4d58-478d-83a5-8b28fef38a2b admin admin] The token cache is being invalidate because project 85a220a727d24cb48b8902f6787479a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81747) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:27:23.137728 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-9c0ddb31-4d58-478d-83a5-8b28fef38a2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:23.138533 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-9c0ddb31-4d58-478d-83a5-8b28fef38a2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:23.142489 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1099/2188] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:27:22 2022] DELETE /identity/v3/projects/85a220a727d24cb48b8902f6787479a5 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:27:23.147233 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ffd2fc-4765-48d2-9950-e89134eb5000 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:23.150146 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ffd2fc-4765-48d2-9950-e89134eb5000 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:23.150146 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ffd2fc-4765-48d2-9950-e89134eb5000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:23.176874 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-80ffd2fc-4765-48d2-9950-e89134eb5000 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:27:23.178416 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-80ffd2fc-4765-48d2-9950-e89134eb5000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:23.309495 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1090/2189] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:27:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:27:23.318913 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6bd3f1-c534-44e7-a2d0-161f50c520e5 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:23.319556 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6bd3f1-c534-44e7-a2d0-161f50c520e5 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:23.319838 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6bd3f1-c534-44e7-a2d0-161f50c520e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:23.362125 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-5b6bd3f1-c534-44e7-a2d0-161f50c520e5 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:27:23.363455 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-5b6bd3f1-c534-44e7-a2d0-161f50c520e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:23.388996 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1100/2190] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:27:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:27:23.390153 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ffd2fc-4765-48d2-9950-e89134eb5000 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:23.410608 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ffd2fc-4765-48d2-9950-e89134eb5000 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:23.426390 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06e5521-696f-4aad-8e25-d39f4ef57b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:23.427116 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06e5521-696f-4aad-8e25-d39f4ef57b57 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:23.427497 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06e5521-696f-4aad-8e25-d39f4ef57b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:23.427844 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06e5521-696f-4aad-8e25-d39f4ef57b57 admin admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:23.428583 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06e5521-696f-4aad-8e25-d39f4ef57b57 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:27:23.430503 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06e5521-696f-4aad-8e25-d39f4ef57b57 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:23.432280 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06e5521-696f-4aad-8e25-d39f4ef57b57 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:23.463182 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1091/2191] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Oct 12 00:27:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:27:23.467333 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6bd3f1-c534-44e7-a2d0-161f50c520e5 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:23.480875 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6bd3f1-c534-44e7-a2d0-161f50c520e5 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:23.490233 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6bab11-96d1-438f-925b-04c6e62107fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:23.491244 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6bab11-96d1-438f-925b-04c6e62107fc admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:23.492493 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6bab11-96d1-438f-925b-04c6e62107fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:23.492962 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6bab11-96d1-438f-925b-04c6e62107fc admin admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:23.493846 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6bab11-96d1-438f-925b-04c6e62107fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:23.497342 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6bab11-96d1-438f-925b-04c6e62107fc admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:23.668304 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1101/2192] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:27:23 2022] POST /identity/v3/projects => generated 373 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:27:23.669750 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06e5521-696f-4aad-8e25-d39f4ef57b57 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:23.685475 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06e5521-696f-4aad-8e25-d39f4ef57b57 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:23.825582 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e59a49-029d-48a9-af69-8f98218a2c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:23.826470 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e59a49-029d-48a9-af69-8f98218a2c31 admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:23.826755 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e59a49-029d-48a9-af69-8f98218a2c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:23.827049 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e59a49-029d-48a9-af69-8f98218a2c31 admin admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:23.827735 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e59a49-029d-48a9-af69-8f98218a2c31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:23.830027 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e59a49-029d-48a9-af69-8f98218a2c31 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:23.989063 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1092/2193] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:27:23 2022] POST /identity/v3/users => generated 287 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:27:23.993493 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6bab11-96d1-438f-925b-04c6e62107fc admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:24.007302 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6bab11-96d1-438f-925b-04c6e62107fc admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:24.021089 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2719c327-ffde-4be1-8669-45a248e3fb75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:24.021731 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2719c327-ffde-4be1-8669-45a248e3fb75 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:24.022010 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2719c327-ffde-4be1-8669-45a248e3fb75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:24.022285 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2719c327-ffde-4be1-8669-45a248e3fb75 admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:24.022954 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2719c327-ffde-4be1-8669-45a248e3fb75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:24.024678 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2719c327-ffde-4be1-8669-45a248e3fb75 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:24.044036 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e59a49-029d-48a9-af69-8f98218a2c31 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:24.045060 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1102/2194] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:27:23 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:27:24.057147 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e59a49-029d-48a9-af69-8f98218a2c31 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:24.072747 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06321206-ed78-4c18-9dc9-e304bdc298e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:24.073410 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06321206-ed78-4c18-9dc9-e304bdc298e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:24.073696 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06321206-ed78-4c18-9dc9-e304bdc298e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:24.073977 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06321206-ed78-4c18-9dc9-e304bdc298e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:24.074793 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06321206-ed78-4c18-9dc9-e304bdc298e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:24.077340 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06321206-ed78-4c18-9dc9-e304bdc298e5 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:24.095449 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1093/2195] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:27:24 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 00:27:24.098448 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2719c327-ffde-4be1-8669-45a248e3fb75 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:24.115197 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2719c327-ffde-4be1-8669-45a248e3fb75 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:24.138212 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e453a47-a8e6-4063-8a33-1398b160cbe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:24.139961 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e453a47-a8e6-4063-8a33-1398b160cbe8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:24.140399 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e453a47-a8e6-4063-8a33-1398b160cbe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:24.140829 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e453a47-a8e6-4063-8a33-1398b160cbe8 admin admin] PATH_INFO: `/v3/projects/f2d4c8d338614442b692b20f3bcb87f9/users/236491bea40f48839993bd1aa4f53dd2/roles/ef689f96442945e8a7bc8cbfee13d909` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:24.187193 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e453a47-a8e6-4063-8a33-1398b160cbe8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2d4c8d338614442b692b20f3bcb87f9, user_id=236491bea40f48839993bd1aa4f53dd2, role_id=ef689f96442945e8a7bc8cbfee13d909)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:24.191867 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e453a47-a8e6-4063-8a33-1398b160cbe8 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:24.279156 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a9106c-684e-4cfd-8b28-073b8805569b None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:24.281270 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a9106c-684e-4cfd-8b28-073b8805569b None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:24.281945 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a9106c-684e-4cfd-8b28-073b8805569b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:24.293640 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1103/2196] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:27:24 2022] PUT /identity/v3/projects/f2d4c8d338614442b692b20f3bcb87f9/users/236491bea40f48839993bd1aa4f53dd2/roles/ef689f96442945e8a7bc8cbfee13d909 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:27:24.303323 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d6a9c7-42cb-4cc2-9eab-e63af2331d38 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:24.303746 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d6a9c7-42cb-4cc2-9eab-e63af2331d38 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:24.304028 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d6a9c7-42cb-4cc2-9eab-e63af2331d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:24.324547 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-55a9106c-684e-4cfd-8b28-073b8805569b None None] MFA Rules not processed for user `d030d18b595144f19d6ad01d4cd5d52b`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:27:24.325997 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-55a9106c-684e-4cfd-8b28-073b8805569b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:24.356705 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-84d6a9c7-42cb-4cc2-9eab-e63af2331d38 None None] MFA Rules not processed for user `236491bea40f48839993bd1aa4f53dd2`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:27:24.358075 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-84d6a9c7-42cb-4cc2-9eab-e63af2331d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:24.404612 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:27:24.427052 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1094/2197] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Oct 12 00:27:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:27:24.440530 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a9106c-684e-4cfd-8b28-073b8805569b None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:24.442238 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-55a9106c-684e-4cfd-8b28-073b8805569b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:24.489024 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1104/2198] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:27:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:27:24.496162 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d6a9c7-42cb-4cc2-9eab-e63af2331d38 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:24.499661 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-84d6a9c7-42cb-4cc2-9eab-e63af2331d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:24.544111 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a9106c-684e-4cfd-8b28-073b8805569b None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:24.555326 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa12790c-6e47-471a-8a84-5cedcf509df9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc1bcffc78844dcbb1a9b7c80285249b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:24.556277 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa12790c-6e47-471a-8a84-5cedcf509df9 service swift] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:24.556408 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa12790c-6e47-471a-8a84-5cedcf509df9 service swift] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:24.557034 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa12790c-6e47-471a-8a84-5cedcf509df9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:24.567796 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa12790c-6e47-471a-8a84-5cedcf509df9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:24.569718 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa12790c-6e47-471a-8a84-5cedcf509df9 service swift] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:24.580096 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1095/2199] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Oct 12 00:27:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 12 00:27:24.597981 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:27:24.628280 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d6a9c7-42cb-4cc2-9eab-e63af2331d38 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:24.634847 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87573444-025f-4326-80d4-6f033041689b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a515b75aa3944912ad2364ffd7924f40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:24.635519 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87573444-025f-4326-80d4-6f033041689b service nova] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:24.635801 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87573444-025f-4326-80d4-6f033041689b service nova] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:24.636073 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87573444-025f-4326-80d4-6f033041689b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:24.646427 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87573444-025f-4326-80d4-6f033041689b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:24.648266 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87573444-025f-4326-80d4-6f033041689b service nova] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:24.658673 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1105/2200] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:27:24 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 00:27:28.761896 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa12790c-6e47-471a-8a84-5cedcf509df9 service swift] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:29.003850 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa12790c-6e47-471a-8a84-5cedcf509df9 service swift] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:29.024815 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6434d80-6916-4655-8f8e-c454ac419e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:29.025686 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6434d80-6916-4655-8f8e-c454ac419e95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:29.026049 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6434d80-6916-4655-8f8e-c454ac419e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:29.026404 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6434d80-6916-4655-8f8e-c454ac419e95 admin admin] PATH_INFO: `/v3/users/236491bea40f48839993bd1aa4f53dd2` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:29.027764 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6434d80-6916-4655-8f8e-c454ac419e95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=236491bea40f48839993bd1aa4f53dd2)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:29.030075 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6434d80-6916-4655-8f8e-c454ac419e95 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:29.270022 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-b6434d80-6916-4655-8f8e-c454ac419e95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '236491bea40f48839993bd1aa4f53dd2'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:29.270787 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-b6434d80-6916-4655-8f8e-c454ac419e95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '236491bea40f48839993bd1aa4f53dd2'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:29.290867 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-b6434d80-6916-4655-8f8e-c454ac419e95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '236491bea40f48839993bd1aa4f53dd2'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:29.387966 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-b6434d80-6916-4655-8f8e-c454ac419e95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '236491bea40f48839993bd1aa4f53dd2'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:29.390931 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-b6434d80-6916-4655-8f8e-c454ac419e95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '236491bea40f48839993bd1aa4f53dd2'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:29.447281 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1096/2201] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:27:28 2022] DELETE /identity/v3/users/236491bea40f48839993bd1aa4f53dd2 => generated 0 bytes in 692 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:27:29.453523 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac81bf31-36c1-474e-ba0c-c7c4c10892f6 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:29.454055 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac81bf31-36c1-474e-ba0c-c7c4c10892f6 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:29.454214 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac81bf31-36c1-474e-ba0c-c7c4c10892f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:29.509087 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-ac81bf31-36c1-474e-ba0c-c7c4c10892f6 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:27:29.510381 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-ac81bf31-36c1-474e-ba0c-c7c4c10892f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:29.748939 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1106/2202] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:27:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:27:29.768932 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6434d80-6916-4655-8f8e-c454ac419e95 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:29.771094 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-b6434d80-6916-4655-8f8e-c454ac419e95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:29.780857 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:27:29.894590 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6434d80-6916-4655-8f8e-c454ac419e95 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:29.899427 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b3d9ef-3152-4cdd-a684-fd0c7dda7ed3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:29.899427 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b3d9ef-3152-4cdd-a684-fd0c7dda7ed3 service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:29.899427 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b3d9ef-3152-4cdd-a684-fd0c7dda7ed3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:29.899792 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b3d9ef-3152-4cdd-a684-fd0c7dda7ed3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:29.911900 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b3d9ef-3152-4cdd-a684-fd0c7dda7ed3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:29.913382 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b3d9ef-3152-4cdd-a684-fd0c7dda7ed3 service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:29.926037 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1097/2203] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:27:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:27:31.428517 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac81bf31-36c1-474e-ba0c-c7c4c10892f6 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:31.430319 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-ac81bf31-36c1-474e-ba0c-c7c4c10892f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:31.453737 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac81bf31-36c1-474e-ba0c-c7c4c10892f6 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:31.459649 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1abe78-cc95-4ae4-90de-32d32d4b07d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:31.460865 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1abe78-cc95-4ae4-90de-32d32d4b07d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:31.461320 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1abe78-cc95-4ae4-90de-32d32d4b07d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:31.461950 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1abe78-cc95-4ae4-90de-32d32d4b07d4 admin admin] PATH_INFO: `/v3/projects/f2d4c8d338614442b692b20f3bcb87f9` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:31.463953 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1abe78-cc95-4ae4-90de-32d32d4b07d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2d4c8d338614442b692b20f3bcb87f9)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:31.467692 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1abe78-cc95-4ae4-90de-32d32d4b07d4 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:31.627551 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-7e1abe78-cc95-4ae4-90de-32d32d4b07d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2d4c8d338614442b692b20f3bcb87f9', 'request_id': 'req-7e1abe78-cc95-4ae4-90de-32d32d4b07d4'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:31.645728 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-7e1abe78-cc95-4ae4-90de-32d32d4b07d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2d4c8d338614442b692b20f3bcb87f9', 'request_id': 'req-7e1abe78-cc95-4ae4-90de-32d32d4b07d4'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:31.718506 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-7e1abe78-cc95-4ae4-90de-32d32d4b07d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2d4c8d338614442b692b20f3bcb87f9', 'request_id': 'req-7e1abe78-cc95-4ae4-90de-32d32d4b07d4'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:31.750466 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-7e1abe78-cc95-4ae4-90de-32d32d4b07d4 admin admin] The token cache is being invalidate because project f2d4c8d338614442b692b20f3bcb87f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81747) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:27:31.752930 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-7e1abe78-cc95-4ae4-90de-32d32d4b07d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:31.753556 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-7e1abe78-cc95-4ae4-90de-32d32d4b07d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:31.763472 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1107/2204] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:27:31 2022] DELETE /identity/v3/projects/f2d4c8d338614442b692b20f3bcb87f9 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:27:31.763873 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d09e19-1df0-40fa-924f-e72c708f3b15 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:31.764336 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d09e19-1df0-40fa-924f-e72c708f3b15 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:31.764567 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d09e19-1df0-40fa-924f-e72c708f3b15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:31.810964 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-95d09e19-1df0-40fa-924f-e72c708f3b15 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:27:31.812805 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-95d09e19-1df0-40fa-924f-e72c708f3b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:31.951331 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1098/2205] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:27:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:27:31.952776 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f1a9ef-baff-466d-b944-4e8e923aa133 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:31.953129 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f1a9ef-baff-466d-b944-4e8e923aa133 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:31.953473 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f1a9ef-baff-466d-b944-4e8e923aa133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:31.969051 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bbe1e1-7e89-4c57-8a5e-9683f1aa6552 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:31.969051 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bbe1e1-7e89-4c57-8a5e-9683f1aa6552 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:31.969051 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bbe1e1-7e89-4c57-8a5e-9683f1aa6552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:32.038294 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-a9bbe1e1-7e89-4c57-8a5e-9683f1aa6552 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:27:32.043202 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-a9bbe1e1-7e89-4c57-8a5e-9683f1aa6552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:32.047036 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-48f1a9ef-baff-466d-b944-4e8e923aa133 None None] MFA Rules not processed for user `269c39f34a014bcc87a1d47d2110105a`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:27:32.052611 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-48f1a9ef-baff-466d-b944-4e8e923aa133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:32.059104 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1099/2206] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:27:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:27:32.060234 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bbe1e1-7e89-4c57-8a5e-9683f1aa6552 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:32.063165 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-a9bbe1e1-7e89-4c57-8a5e-9683f1aa6552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:32.106317 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bbe1e1-7e89-4c57-8a5e-9683f1aa6552 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:32.120909 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:27:32.131699 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190c7439-3583-4be1-a9c5-8d0f702e928f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:32.135509 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c7439-3583-4be1-a9c5-8d0f702e928f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:32.135509 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c7439-3583-4be1-a9c5-8d0f702e928f admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:32.135509 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c7439-3583-4be1-a9c5-8d0f702e928f admin admin] PATH_INFO: `/v3/users/de766cb582ba4c18b1fda9bf4ad43f65` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:32.135509 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190c7439-3583-4be1-a9c5-8d0f702e928f admin admin] RBAC: Authorizing `identity:delete_user(user_id=de766cb582ba4c18b1fda9bf4ad43f65)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:32.136214 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190c7439-3583-4be1-a9c5-8d0f702e928f admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:32.149395 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1108/2207] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:27:31 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:27:32.150695 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f1a9ef-baff-466d-b944-4e8e923aa133 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:32.165010 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f1a9ef-baff-466d-b944-4e8e923aa133 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:32.170602 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5479eb94-4c5b-4629-938e-dac20bf7a2ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:32.171303 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5479eb94-4c5b-4629-938e-dac20bf7a2ce admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:32.171705 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5479eb94-4c5b-4629-938e-dac20bf7a2ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:32.172156 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5479eb94-4c5b-4629-938e-dac20bf7a2ce admin admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:32.172876 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5479eb94-4c5b-4629-938e-dac20bf7a2ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:27:32.173154 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5479eb94-4c5b-4629-938e-dac20bf7a2ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:32.175127 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5479eb94-4c5b-4629-938e-dac20bf7a2ce admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:32.202974 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1109/2208] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Oct 12 00:27:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:27:32.214180 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5479eb94-4c5b-4629-938e-dac20bf7a2ce admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:32.217026 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-5479eb94-4c5b-4629-938e-dac20bf7a2ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:32.338689 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5479eb94-4c5b-4629-938e-dac20bf7a2ce admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:32.345088 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c05e08-ebf5-411c-b6c2-6f9c7d177cee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a515b75aa3944912ad2364ffd7924f40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:32.345925 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c05e08-ebf5-411c-b6c2-6f9c7d177cee service nova] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:32.346209 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c05e08-ebf5-411c-b6c2-6f9c7d177cee service nova] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:32.346523 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c05e08-ebf5-411c-b6c2-6f9c7d177cee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:32.348698 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-00c05e08-ebf5-411c-b6c2-6f9c7d177cee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:32.393798 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-190c7439-3583-4be1-a9c5-8d0f702e928f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de766cb582ba4c18b1fda9bf4ad43f65'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:32.396668 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-190c7439-3583-4be1-a9c5-8d0f702e928f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de766cb582ba4c18b1fda9bf4ad43f65'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:32.442676 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-190c7439-3583-4be1-a9c5-8d0f702e928f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de766cb582ba4c18b1fda9bf4ad43f65'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:32.482185 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c05e08-ebf5-411c-b6c2-6f9c7d177cee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:32.483865 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c05e08-ebf5-411c-b6c2-6f9c7d177cee service nova] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:32.495511 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1110/2209] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:27:32 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:27:32.510691 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c05e08-ebf5-411c-b6c2-6f9c7d177cee service nova] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:32.518411 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-190c7439-3583-4be1-a9c5-8d0f702e928f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de766cb582ba4c18b1fda9bf4ad43f65'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:32.519294 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-190c7439-3583-4be1-a9c5-8d0f702e928f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de766cb582ba4c18b1fda9bf4ad43f65'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:32.539074 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c05e08-ebf5-411c-b6c2-6f9c7d177cee service nova] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:32.545121 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1100/2210] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:27:32 2022] DELETE /identity/v3/users/de766cb582ba4c18b1fda9bf4ad43f65 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:27:32.575134 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678b02c4-28c2-49cf-b159-6063d70c3f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:32.575795 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678b02c4-28c2-49cf-b159-6063d70c3f50 admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:32.576072 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678b02c4-28c2-49cf-b159-6063d70c3f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:32.576344 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678b02c4-28c2-49cf-b159-6063d70c3f50 admin admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:32.576991 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678b02c4-28c2-49cf-b159-6063d70c3f50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:32.579013 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678b02c4-28c2-49cf-b159-6063d70c3f50 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:32.726697 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1111/2211] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:27:32 2022] POST /identity/v3/projects => generated 403 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:27:32.731803 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190c7439-3583-4be1-a9c5-8d0f702e928f admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:32.734239 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-190c7439-3583-4be1-a9c5-8d0f702e928f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:32.760644 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:27:32.891592 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190c7439-3583-4be1-a9c5-8d0f702e928f admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:32.986306 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678b02c4-28c2-49cf-b159-6063d70c3f50 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:32.989726 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-678b02c4-28c2-49cf-b159-6063d70c3f50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:33.019576 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d4d378-1834-47d7-a34e-23c1df97eb82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:33.021010 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d4d378-1834-47d7-a34e-23c1df97eb82 admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:33.021164 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d4d378-1834-47d7-a34e-23c1df97eb82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:33.023744 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d4d378-1834-47d7-a34e-23c1df97eb82 admin admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:33.024818 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d4d378-1834-47d7-a34e-23c1df97eb82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:33.027340 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d4d378-1834-47d7-a34e-23c1df97eb82 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:33.145332 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:27:33.162559 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1101/2212] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:27:32 2022] POST /identity/v3/users => generated 302 bytes in 431 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:27:33.166395 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d4d378-1834-47d7-a34e-23c1df97eb82 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:33.185764 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d4d378-1834-47d7-a34e-23c1df97eb82 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:33.189677 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678b02c4-28c2-49cf-b159-6063d70c3f50 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:33.193278 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c76c5e3-0ef5-4913-bf95-dc1e80ba4d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:33.194196 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c76c5e3-0ef5-4913-bf95-dc1e80ba4d15 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:33.194757 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c76c5e3-0ef5-4913-bf95-dc1e80ba4d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:33.195319 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c76c5e3-0ef5-4913-bf95-dc1e80ba4d15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:33.196464 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c76c5e3-0ef5-4913-bf95-dc1e80ba4d15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:33.199339 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c76c5e3-0ef5-4913-bf95-dc1e80ba4d15 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:33.206465 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce64a8b2-3c78-49bd-89a2-f43739b041da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:33.207137 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce64a8b2-3c78-49bd-89a2-f43739b041da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:33.207482 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce64a8b2-3c78-49bd-89a2-f43739b041da admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:33.207702 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce64a8b2-3c78-49bd-89a2-f43739b041da admin admin] PATH_INFO: `/v3/projects/10160596b45c49249155b0c2585e70bb` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:33.213534 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce64a8b2-3c78-49bd-89a2-f43739b041da admin admin] RBAC: Authorizing `identity:delete_project(project_id=10160596b45c49249155b0c2585e70bb)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:33.220220 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1102/2213] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:27:33 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:27:33.221345 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce64a8b2-3c78-49bd-89a2-f43739b041da admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:33.228208 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c76c5e3-0ef5-4913-bf95-dc1e80ba4d15 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:33.261104 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c76c5e3-0ef5-4913-bf95-dc1e80ba4d15 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:33.274623 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ccd6ad-a3ff-4b30-b9f0-13f92d7fe53b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:33.275611 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ccd6ad-a3ff-4b30-b9f0-13f92d7fe53b admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:33.276163 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ccd6ad-a3ff-4b30-b9f0-13f92d7fe53b admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:33.276750 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ccd6ad-a3ff-4b30-b9f0-13f92d7fe53b admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:33.277633 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ccd6ad-a3ff-4b30-b9f0-13f92d7fe53b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:33.280513 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ccd6ad-a3ff-4b30-b9f0-13f92d7fe53b admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:33.298598 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1103/2214] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:27:33 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:27:33.303620 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ccd6ad-a3ff-4b30-b9f0-13f92d7fe53b admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:33.315898 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ccd6ad-a3ff-4b30-b9f0-13f92d7fe53b admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:33.323871 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8660542d-5bb2-4708-8c2b-6584fa900b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:33.324618 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8660542d-5bb2-4708-8c2b-6584fa900b8e admin admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:33.324904 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8660542d-5bb2-4708-8c2b-6584fa900b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:33.325337 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8660542d-5bb2-4708-8c2b-6584fa900b8e admin admin] PATH_INFO: `/v3/projects/0a0c55efa4854edd898655d0d2e5cbd8/users/ed328986068b4755acf9fe6f3697a40e/roles/ef689f96442945e8a7bc8cbfee13d909` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:33.346421 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8660542d-5bb2-4708-8c2b-6584fa900b8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a0c55efa4854edd898655d0d2e5cbd8, user_id=ed328986068b4755acf9fe6f3697a40e, role_id=ef689f96442945e8a7bc8cbfee13d909)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:33.349810 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8660542d-5bb2-4708-8c2b-6584fa900b8e admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:33.421569 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1104/2215] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:27:33 2022] PUT /identity/v3/projects/0a0c55efa4854edd898655d0d2e5cbd8/users/ed328986068b4755acf9fe6f3697a40e/roles/ef689f96442945e8a7bc8cbfee13d909 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:27:33.427317 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5e335a-f850-4618-9e17-2f6bd18911d5 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:33.427699 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5e335a-f850-4618-9e17-2f6bd18911d5 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:33.427975 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5e335a-f850-4618-9e17-2f6bd18911d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:33.436829 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-ce64a8b2-3c78-49bd-89a2-f43739b041da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10160596b45c49249155b0c2585e70bb', 'request_id': 'req-ce64a8b2-3c78-49bd-89a2-f43739b041da'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:33.451958 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-ce64a8b2-3c78-49bd-89a2-f43739b041da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10160596b45c49249155b0c2585e70bb', 'request_id': 'req-ce64a8b2-3c78-49bd-89a2-f43739b041da'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:33.494780 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-8c5e335a-f850-4618-9e17-2f6bd18911d5 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:27:33.500982 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-8c5e335a-f850-4618-9e17-2f6bd18911d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:33.529514 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-ce64a8b2-3c78-49bd-89a2-f43739b041da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10160596b45c49249155b0c2585e70bb', 'request_id': 'req-ce64a8b2-3c78-49bd-89a2-f43739b041da'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:33.579197 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-ce64a8b2-3c78-49bd-89a2-f43739b041da admin admin] The token cache is being invalidate because project 10160596b45c49249155b0c2585e70bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81747) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:27:33.582747 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-ce64a8b2-3c78-49bd-89a2-f43739b041da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:33.586154 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-ce64a8b2-3c78-49bd-89a2-f43739b041da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:33.602015 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1112/2216] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:27:32 2022] DELETE /identity/v3/projects/10160596b45c49249155b0c2585e70bb => generated 0 bytes in 610 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 00:27:33.616397 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919420f0-25f7-4029-a34d-83f5fc405553 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:33.617344 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919420f0-25f7-4029-a34d-83f5fc405553 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:33.618086 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919420f0-25f7-4029-a34d-83f5fc405553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:33.661048 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1105/2217] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:27:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 00:27:33.666520 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-919420f0-25f7-4029-a34d-83f5fc405553 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:27:33.668546 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5e335a-f850-4618-9e17-2f6bd18911d5 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:33.674409 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-919420f0-25f7-4029-a34d-83f5fc405553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:33.674541 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-8c5e335a-f850-4618-9e17-2f6bd18911d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:33.679249 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1113/2218] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:27:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:27:33.688102 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba9db3e-b4e1-45e5-9a14-db3b0a3131cd None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:33.688657 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba9db3e-b4e1-45e5-9a14-db3b0a3131cd None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:33.689060 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:27:33.689268 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba9db3e-b4e1-45e5-9a14-db3b0a3131cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:33.718745 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-dba9db3e-b4e1-45e5-9a14-db3b0a3131cd None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:27:33.720425 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-dba9db3e-b4e1-45e5-9a14-db3b0a3131cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:33.748443 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1114/2219] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:27:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 00:27:33.749923 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba9db3e-b4e1-45e5-9a14-db3b0a3131cd None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:33.773609 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba9db3e-b4e1-45e5-9a14-db3b0a3131cd None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:33.783754 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7299d92d-a7c4-4e96-8af3-f6c7181e26fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:33.785097 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7299d92d-a7c4-4e96-8af3-f6c7181e26fd admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:33.785434 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7299d92d-a7c4-4e96-8af3-f6c7181e26fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:33.785764 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7299d92d-a7c4-4e96-8af3-f6c7181e26fd admin admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:33.786511 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7299d92d-a7c4-4e96-8af3-f6c7181e26fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:27:33.786873 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7299d92d-a7c4-4e96-8af3-f6c7181e26fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:33.788773 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7299d92d-a7c4-4e96-8af3-f6c7181e26fd admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:33.803493 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1115/2220] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Oct 12 00:27:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:27:33.809028 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7299d92d-a7c4-4e96-8af3-f6c7181e26fd admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:33.821030 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7299d92d-a7c4-4e96-8af3-f6c7181e26fd admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:33.827183 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfdb810-fa87-4827-9986-ce6142f9cdb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:33.828136 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfdb810-fa87-4827-9986-ce6142f9cdb3 admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:33.828712 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfdb810-fa87-4827-9986-ce6142f9cdb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:33.829344 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfdb810-fa87-4827-9986-ce6142f9cdb3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:33.830157 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfdb810-fa87-4827-9986-ce6142f9cdb3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:33.832678 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfdb810-fa87-4827-9986-ce6142f9cdb3 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:33.835656 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5e335a-f850-4618-9e17-2f6bd18911d5 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:33.840601 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb3c9dc-b243-44e5-9a83-5f07f9b9342a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:33.844522 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb3c9dc-b243-44e5-9a83-5f07f9b9342a service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:33.844522 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb3c9dc-b243-44e5-9a83-5f07f9b9342a service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:33.844522 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb3c9dc-b243-44e5-9a83-5f07f9b9342a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:33.857234 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb3c9dc-b243-44e5-9a83-5f07f9b9342a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:33.860281 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb3c9dc-b243-44e5-9a83-5f07f9b9342a service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:33.906987 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1106/2221] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:27:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:27:33.911456 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1116/2222] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:27:33 2022] POST /identity/v3/projects => generated 379 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 00:27:33.915950 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb3c9dc-b243-44e5-9a83-5f07f9b9342a service neutron] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:33.942791 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb3c9dc-b243-44e5-9a83-5f07f9b9342a service neutron] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:34.076076 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f2a78c-1ac4-4217-b8bd-dd1d73930625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:34.076802 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f2a78c-1ac4-4217-b8bd-dd1d73930625 admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:34.077163 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f2a78c-1ac4-4217-b8bd-dd1d73930625 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:34.079627 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f2a78c-1ac4-4217-b8bd-dd1d73930625 admin admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:34.085575 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f2a78c-1ac4-4217-b8bd-dd1d73930625 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:34.085575 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f2a78c-1ac4-4217-b8bd-dd1d73930625 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:34.249090 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1107/2223] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:27:33 2022] POST /identity/v3/users => generated 290 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:27:34.255615 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfdb810-fa87-4827-9986-ce6142f9cdb3 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:34.273222 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfdb810-fa87-4827-9986-ce6142f9cdb3 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:34.278411 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077310d4-7139-4c86-abcc-ca24a98054b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:34.280816 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077310d4-7139-4c86-abcc-ca24a98054b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:34.280816 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077310d4-7139-4c86-abcc-ca24a98054b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:34.280816 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077310d4-7139-4c86-abcc-ca24a98054b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:34.280816 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077310d4-7139-4c86-abcc-ca24a98054b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:34.282316 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077310d4-7139-4c86-abcc-ca24a98054b3 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:34.313251 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1117/2224] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:27:34 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:27:34.317592 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f2a78c-1ac4-4217-b8bd-dd1d73930625 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:34.335190 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f2a78c-1ac4-4217-b8bd-dd1d73930625 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:34.344023 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef6b26b-dbd2-4446-b366-320d500f01c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:34.348012 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef6b26b-dbd2-4446-b366-320d500f01c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:34.348378 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef6b26b-dbd2-4446-b366-320d500f01c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:34.348709 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef6b26b-dbd2-4446-b366-320d500f01c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:34.349451 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef6b26b-dbd2-4446-b366-320d500f01c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:34.351458 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef6b26b-dbd2-4446-b366-320d500f01c0 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:34.383790 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1108/2225] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:27:34 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:27:34.389166 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077310d4-7139-4c86-abcc-ca24a98054b3 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:34.402481 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077310d4-7139-4c86-abcc-ca24a98054b3 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:34.408183 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0ca3fa-8964-43a8-83ba-d485fd89b8e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:34.409692 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0ca3fa-8964-43a8-83ba-d485fd89b8e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:34.410031 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0ca3fa-8964-43a8-83ba-d485fd89b8e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:34.410307 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0ca3fa-8964-43a8-83ba-d485fd89b8e1 admin admin] PATH_INFO: `/v3/projects/fff997542a6949eeb929bc0bef72c20e/users/ede8ec1937224e9cae8d619435699ef0/roles/ef689f96442945e8a7bc8cbfee13d909` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:34.443959 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0ca3fa-8964-43a8-83ba-d485fd89b8e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fff997542a6949eeb929bc0bef72c20e, user_id=ede8ec1937224e9cae8d619435699ef0, role_id=ef689f96442945e8a7bc8cbfee13d909)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:34.446716 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0ca3fa-8964-43a8-83ba-d485fd89b8e1 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:34.793062 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1118/2226] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:27:34 2022] PUT /identity/v3/projects/fff997542a6949eeb929bc0bef72c20e/users/ede8ec1937224e9cae8d619435699ef0/roles/ef689f96442945e8a7bc8cbfee13d909 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 00:27:34.805766 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b4090c-7aa3-4f9d-8027-1680fac24200 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:34.806253 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b4090c-7aa3-4f9d-8027-1680fac24200 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:34.806630 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b4090c-7aa3-4f9d-8027-1680fac24200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:34.849293 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-a5b4090c-7aa3-4f9d-8027-1680fac24200 None None] MFA Rules not processed for user `ede8ec1937224e9cae8d619435699ef0`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:27:34.853291 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-a5b4090c-7aa3-4f9d-8027-1680fac24200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:34.981189 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1109/2227] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:27:34 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 00:27:34.988939 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0ca3fa-8964-43a8-83ba-d485fd89b8e1 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:35.094325 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:27:35.105064 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0ca3fa-8964-43a8-83ba-d485fd89b8e1 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:35.114609 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f67b22-22ed-4974-800e-1a8b55a8b42b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:35.115780 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f67b22-22ed-4974-800e-1a8b55a8b42b admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:35.116550 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f67b22-22ed-4974-800e-1a8b55a8b42b admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:35.117098 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f67b22-22ed-4974-800e-1a8b55a8b42b admin admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:35.118403 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f67b22-22ed-4974-800e-1a8b55a8b42b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:35.123076 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f67b22-22ed-4974-800e-1a8b55a8b42b admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:35.268879 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1119/2228] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:27:34 2022] POST /identity/v3/projects => generated 381 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:27:35.269521 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b4090c-7aa3-4f9d-8027-1680fac24200 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:35.282132 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b4090c-7aa3-4f9d-8027-1680fac24200 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:35.293711 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:27:35.370779 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d938e2-1258-43cf-ae6e-9e0a7536e581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:35.371566 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d938e2-1258-43cf-ae6e-9e0a7536e581 admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:35.371897 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d938e2-1258-43cf-ae6e-9e0a7536e581 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:35.372148 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d938e2-1258-43cf-ae6e-9e0a7536e581 admin admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:35.373066 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d938e2-1258-43cf-ae6e-9e0a7536e581 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:35.375513 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d938e2-1258-43cf-ae6e-9e0a7536e581 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:35.509091 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1110/2229] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:27:35 2022] POST /identity/v3/users => generated 297 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 00:27:35.513138 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f67b22-22ed-4974-800e-1a8b55a8b42b admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:35.526568 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f67b22-22ed-4974-800e-1a8b55a8b42b admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:35.533115 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6028b35f-8270-4dfc-baec-4d03efff58b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:35.544617 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6028b35f-8270-4dfc-baec-4d03efff58b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:35.544617 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6028b35f-8270-4dfc-baec-4d03efff58b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:35.544617 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6028b35f-8270-4dfc-baec-4d03efff58b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:35.544617 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6028b35f-8270-4dfc-baec-4d03efff58b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:35.544617 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6028b35f-8270-4dfc-baec-4d03efff58b5 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:35.565983 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1120/2230] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:27:35 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:27:35.571597 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d938e2-1258-43cf-ae6e-9e0a7536e581 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:35.593566 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d938e2-1258-43cf-ae6e-9e0a7536e581 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:35.604032 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf8548b-f3f4-4f17-8dbd-3eb1b1689fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:35.605991 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf8548b-f3f4-4f17-8dbd-3eb1b1689fb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:35.606529 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf8548b-f3f4-4f17-8dbd-3eb1b1689fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:35.606862 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf8548b-f3f4-4f17-8dbd-3eb1b1689fb2 admin admin] PATH_INFO: `/v3/projects/852075b062c54374b402e784c2cd19ea/users/8187f9d5817148f29c9a52ed71ee6306/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:35.646508 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf8548b-f3f4-4f17-8dbd-3eb1b1689fb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=852075b062c54374b402e784c2cd19ea, user_id=8187f9d5817148f29c9a52ed71ee6306, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:35.658065 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf8548b-f3f4-4f17-8dbd-3eb1b1689fb2 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:35.802498 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1111/2231] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:27:35 2022] PUT /identity/v3/projects/852075b062c54374b402e784c2cd19ea/users/8187f9d5817148f29c9a52ed71ee6306/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:27:35.803821 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6028b35f-8270-4dfc-baec-4d03efff58b5 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:35.823295 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6028b35f-8270-4dfc-baec-4d03efff58b5 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:35.986797 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324b5043-d259-4622-8a5f-b0245c0fad23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:35.991644 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324b5043-d259-4622-8a5f-b0245c0fad23 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:35.991644 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324b5043-d259-4622-8a5f-b0245c0fad23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:35.991644 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324b5043-d259-4622-8a5f-b0245c0fad23 admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:35.991644 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324b5043-d259-4622-8a5f-b0245c0fad23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:35.991644 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324b5043-d259-4622-8a5f-b0245c0fad23 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:36.015835 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e1acdd-28a7-40a7-a7b3-a1c0e57fd4ea None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:36.015835 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e1acdd-28a7-40a7-a7b3-a1c0e57fd4ea None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:36.019481 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e1acdd-28a7-40a7-a7b3-a1c0e57fd4ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:36.019764 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1121/2232] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:27:35 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:27:36.025293 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324b5043-d259-4622-8a5f-b0245c0fad23 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:36.052934 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324b5043-d259-4622-8a5f-b0245c0fad23 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:36.060274 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439f255a-bfef-44d6-8ea5-08291e3b2fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:36.061074 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439f255a-bfef-44d6-8ea5-08291e3b2fb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:36.061463 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439f255a-bfef-44d6-8ea5-08291e3b2fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:36.062222 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439f255a-bfef-44d6-8ea5-08291e3b2fb1 admin admin] PATH_INFO: `/v3/projects/852075b062c54374b402e784c2cd19ea/users/8187f9d5817148f29c9a52ed71ee6306/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:36.067108 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439f255a-bfef-44d6-8ea5-08291e3b2fb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=852075b062c54374b402e784c2cd19ea, user_id=8187f9d5817148f29c9a52ed71ee6306, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:36.075643 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439f255a-bfef-44d6-8ea5-08291e3b2fb1 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:36.077253 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-99e1acdd-28a7-40a7-a7b3-a1c0e57fd4ea None None] MFA Rules not processed for user `ed328986068b4755acf9fe6f3697a40e`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:27:36.079604 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-99e1acdd-28a7-40a7-a7b3-a1c0e57fd4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:36.102308 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1122/2233] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:27:36 2022] PUT /identity/v3/projects/852075b062c54374b402e784c2cd19ea/users/8187f9d5817148f29c9a52ed71ee6306/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:27:36.104739 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee738a-be04-4470-a0a3-5705afd95b86 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:36.105312 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee738a-be04-4470-a0a3-5705afd95b86 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:36.105899 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee738a-be04-4470-a0a3-5705afd95b86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:36.277786 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-56ee738a-be04-4470-a0a3-5705afd95b86 None None] MFA Rules not processed for user `8187f9d5817148f29c9a52ed71ee6306`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:27:36.277786 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-56ee738a-be04-4470-a0a3-5705afd95b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:36.278453 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1112/2234] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:27:36 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:27:36.278657 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1123/2235] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:27:36 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:27:36.292483 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e1acdd-28a7-40a7-a7b3-a1c0e57fd4ea None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:36.296507 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-99e1acdd-28a7-40a7-a7b3-a1c0e57fd4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:36.305038 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ee738a-be04-4470-a0a3-5705afd95b86 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:36.307070 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-56ee738a-be04-4470-a0a3-5705afd95b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:36.376203 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e1acdd-28a7-40a7-a7b3-a1c0e57fd4ea None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:36.378612 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ee738a-be04-4470-a0a3-5705afd95b86 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:36.382969 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4ed75c-d38d-43f8-af1c-5c1317b3d442 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:36.383175 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12505cd-fef6-4e67-a395-ad6d95ae6f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:36.383636 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4ed75c-d38d-43f8-af1c-5c1317b3d442 service cinder] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:36.383918 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4ed75c-d38d-43f8-af1c-5c1317b3d442 service cinder] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:36.384221 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4ed75c-d38d-43f8-af1c-5c1317b3d442 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:36.384338 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12505cd-fef6-4e67-a395-ad6d95ae6f70 admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:36.384582 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12505cd-fef6-4e67-a395-ad6d95ae6f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:36.384858 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12505cd-fef6-4e67-a395-ad6d95ae6f70 admin admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:36.385725 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12505cd-fef6-4e67-a395-ad6d95ae6f70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:36.388296 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12505cd-fef6-4e67-a395-ad6d95ae6f70 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:36.398003 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4ed75c-d38d-43f8-af1c-5c1317b3d442 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:36.399480 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4ed75c-d38d-43f8-af1c-5c1317b3d442 service cinder] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:36.410928 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:27:36.412267 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1124/2236] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:27:36 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:27:37.246458 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1113/2237] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:27:36 2022] POST /identity/v3/projects => generated 401 bytes in 954 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:27:37.253005 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4ed75c-d38d-43f8-af1c-5c1317b3d442 service cinder] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:37.276825 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4ed75c-d38d-43f8-af1c-5c1317b3d442 service cinder] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:37.314064 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:27:37.337569 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30405c3f-8734-4938-b1bc-0f609fa25251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:37.338344 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30405c3f-8734-4938-b1bc-0f609fa25251 admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:37.338618 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30405c3f-8734-4938-b1bc-0f609fa25251 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:37.338883 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30405c3f-8734-4938-b1bc-0f609fa25251 admin admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:37.339558 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30405c3f-8734-4938-b1bc-0f609fa25251 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:37.341468 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30405c3f-8734-4938-b1bc-0f609fa25251 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:37.354774 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e106d6ed-69d0-4285-a224-cff529ab2848 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:37.354896 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e106d6ed-69d0-4285-a224-cff529ab2848 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:37.355122 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e106d6ed-69d0-4285-a224-cff529ab2848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:37.380000 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-e106d6ed-69d0-4285-a224-cff529ab2848 None None] MFA Rules not processed for user `da2a3fcb0287474cb93238923aa9c7c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:27:37.381244 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-e106d6ed-69d0-4285-a224-cff529ab2848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:37.435561 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1125/2238] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:27:37 2022] POST /identity/v3/users => generated 307 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:27:37.435561 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30405c3f-8734-4938-b1bc-0f609fa25251 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:37.439495 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1114/2239] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:27:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 00:27:37.440386 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e106d6ed-69d0-4285-a224-cff529ab2848 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:37.451549 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e106d6ed-69d0-4285-a224-cff529ab2848 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:37.456250 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2be4b1-c5fc-4854-9a7c-8fae0177c955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:37.456911 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2be4b1-c5fc-4854-9a7c-8fae0177c955 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:37.457127 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2be4b1-c5fc-4854-9a7c-8fae0177c955 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:37.457776 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2be4b1-c5fc-4854-9a7c-8fae0177c955 admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:37.458043 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2be4b1-c5fc-4854-9a7c-8fae0177c955 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:37.459676 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2be4b1-c5fc-4854-9a7c-8fae0177c955 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:37.469738 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1115/2240] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:27:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:27:37.480086 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2be4b1-c5fc-4854-9a7c-8fae0177c955 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:37.492297 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2be4b1-c5fc-4854-9a7c-8fae0177c955 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:37.495170 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30405c3f-8734-4938-b1bc-0f609fa25251 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:37.497310 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bd138c-afef-48e9-ad00-d4057c97b0c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:37.498161 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd138c-afef-48e9-ad00-d4057c97b0c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:37.498429 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd138c-afef-48e9-ad00-d4057c97b0c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:37.498785 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd138c-afef-48e9-ad00-d4057c97b0c5 admin admin] PATH_INFO: `/v3/projects/13f610c86c4541208de543b8cfcf8ede/users/48e06cb16565440dac6d090e25cd1802/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:37.500154 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62af8aba-de9f-460c-acf5-ad6969a16e15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:37.500869 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62af8aba-de9f-460c-acf5-ad6969a16e15 service cinder] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:37.501259 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62af8aba-de9f-460c-acf5-ad6969a16e15 service cinder] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:37.501627 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62af8aba-de9f-460c-acf5-ad6969a16e15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:37.514900 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bd138c-afef-48e9-ad00-d4057c97b0c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13f610c86c4541208de543b8cfcf8ede, user_id=48e06cb16565440dac6d090e25cd1802, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:37.517856 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bd138c-afef-48e9-ad00-d4057c97b0c5 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:37.570625 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62af8aba-de9f-460c-acf5-ad6969a16e15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:37.572119 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62af8aba-de9f-460c-acf5-ad6969a16e15 service cinder] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:37.581524 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1126/2241] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:27:37 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:27:37.601152 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1116/2242] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:27:37 2022] PUT /identity/v3/projects/13f610c86c4541208de543b8cfcf8ede/users/48e06cb16565440dac6d090e25cd1802/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:27:37.605471 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62af8aba-de9f-460c-acf5-ad6969a16e15 service cinder] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:37.623994 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62af8aba-de9f-460c-acf5-ad6969a16e15 service cinder] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:37.690219 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf768e9-29e6-4af1-af0c-782591c2c8e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:37.690885 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf768e9-29e6-4af1-af0c-782591c2c8e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:37.691108 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf768e9-29e6-4af1-af0c-782591c2c8e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:37.691442 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf768e9-29e6-4af1-af0c-782591c2c8e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:37.692045 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf768e9-29e6-4af1-af0c-782591c2c8e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:37.693973 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf768e9-29e6-4af1-af0c-782591c2c8e5 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:37.707405 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1127/2243] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:27:37 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:27:37.711716 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bd138c-afef-48e9-ad00-d4057c97b0c5 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:37.722228 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bd138c-afef-48e9-ad00-d4057c97b0c5 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:37.727081 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64726be3-abda-404a-a366-c99e19b300e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:37.727824 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64726be3-abda-404a-a366-c99e19b300e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:37.728106 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64726be3-abda-404a-a366-c99e19b300e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:37.728375 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64726be3-abda-404a-a366-c99e19b300e7 admin admin] PATH_INFO: `/v3/projects/13f610c86c4541208de543b8cfcf8ede/users/48e06cb16565440dac6d090e25cd1802/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:37.730330 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64726be3-abda-404a-a366-c99e19b300e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13f610c86c4541208de543b8cfcf8ede, user_id=48e06cb16565440dac6d090e25cd1802, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:37.733038 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64726be3-abda-404a-a366-c99e19b300e7 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:37.744834 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1117/2244] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:27:37 2022] PUT /identity/v3/projects/13f610c86c4541208de543b8cfcf8ede/users/48e06cb16565440dac6d090e25cd1802/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:27:37.922536 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:27:39.049554 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8345815b-55c5-4ce5-a09b-e5e71c127db3 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:39.050199 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8345815b-55c5-4ce5-a09b-e5e71c127db3 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:39.050199 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8345815b-55c5-4ce5-a09b-e5e71c127db3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:39.086546 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-8345815b-55c5-4ce5-a09b-e5e71c127db3 None None] MFA Rules not processed for user `48e06cb16565440dac6d090e25cd1802`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:27:39.087792 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-8345815b-55c5-4ce5-a09b-e5e71c127db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:39.179329 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1128/2245] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:27:39 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:27:40.724962 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64726be3-abda-404a-a366-c99e19b300e7 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:40.728592 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-64726be3-abda-404a-a366-c99e19b300e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:40.813323 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64726be3-abda-404a-a366-c99e19b300e7 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:40.815765 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a81bf3d-f1f0-4cdb-8af3-55ed92bc7c31 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e528da752ed4c78bc747215909f6569', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:40.816407 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a81bf3d-f1f0-4cdb-8af3-55ed92bc7c31 service placement] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:40.816745 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a81bf3d-f1f0-4cdb-8af3-55ed92bc7c31 service placement] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:40.817024 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a81bf3d-f1f0-4cdb-8af3-55ed92bc7c31 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:40.819238 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-9a81bf3d-f1f0-4cdb-8af3-55ed92bc7c31 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:40.832774 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a81bf3d-f1f0-4cdb-8af3-55ed92bc7c31 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:40.834374 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a81bf3d-f1f0-4cdb-8af3-55ed92bc7c31 service placement] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:40.845402 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1118/2246] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:27:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:27:40.908186 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8345815b-55c5-4ce5-a09b-e5e71c127db3 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:40.979921 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8345815b-55c5-4ce5-a09b-e5e71c127db3 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:40.984579 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4502a89-28b6-4786-9650-86402cfd545b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:40.985214 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4502a89-28b6-4786-9650-86402cfd545b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:40.985493 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4502a89-28b6-4786-9650-86402cfd545b admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:40.985764 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4502a89-28b6-4786-9650-86402cfd545b admin admin] PATH_INFO: `/v3/users/ed328986068b4755acf9fe6f3697a40e` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:40.986921 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4502a89-28b6-4786-9650-86402cfd545b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed328986068b4755acf9fe6f3697a40e)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:40.989151 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4502a89-28b6-4786-9650-86402cfd545b admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:41.170188 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-d4502a89-28b6-4786-9650-86402cfd545b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed328986068b4755acf9fe6f3697a40e'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:41.170840 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-d4502a89-28b6-4786-9650-86402cfd545b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed328986068b4755acf9fe6f3697a40e'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:41.171983 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:27:41.185187 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-d4502a89-28b6-4786-9650-86402cfd545b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed328986068b4755acf9fe6f3697a40e'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:41.238491 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-d4502a89-28b6-4786-9650-86402cfd545b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed328986068b4755acf9fe6f3697a40e'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:41.239095 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-d4502a89-28b6-4786-9650-86402cfd545b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed328986068b4755acf9fe6f3697a40e'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:41.252071 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1129/2247] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:27:40 2022] DELETE /identity/v3/users/ed328986068b4755acf9fe6f3697a40e => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:27:41.521160 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a81bf3d-f1f0-4cdb-8af3-55ed92bc7c31 service placement] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:41.546621 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a81bf3d-f1f0-4cdb-8af3-55ed92bc7c31 service placement] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:41.595282 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:27:41.619567 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f275d50f-ded6-44da-81dc-307e15a8480a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:41.620666 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f275d50f-ded6-44da-81dc-307e15a8480a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:41.620958 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f275d50f-ded6-44da-81dc-307e15a8480a admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:41.621327 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f275d50f-ded6-44da-81dc-307e15a8480a admin admin] PATH_INFO: `/v3/projects/0a0c55efa4854edd898655d0d2e5cbd8` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:41.622551 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f275d50f-ded6-44da-81dc-307e15a8480a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a0c55efa4854edd898655d0d2e5cbd8)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:41.624686 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f275d50f-ded6-44da-81dc-307e15a8480a admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:41.744756 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-f275d50f-ded6-44da-81dc-307e15a8480a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a0c55efa4854edd898655d0d2e5cbd8', 'request_id': 'req-f275d50f-ded6-44da-81dc-307e15a8480a'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:41.752730 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-f275d50f-ded6-44da-81dc-307e15a8480a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a0c55efa4854edd898655d0d2e5cbd8', 'request_id': 'req-f275d50f-ded6-44da-81dc-307e15a8480a'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:41.848930 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-f275d50f-ded6-44da-81dc-307e15a8480a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a0c55efa4854edd898655d0d2e5cbd8', 'request_id': 'req-f275d50f-ded6-44da-81dc-307e15a8480a'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:41.861067 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-f275d50f-ded6-44da-81dc-307e15a8480a admin admin] The token cache is being invalidate because project 0a0c55efa4854edd898655d0d2e5cbd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81746) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:27:41.861361 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-f275d50f-ded6-44da-81dc-307e15a8480a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:41.862658 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-f275d50f-ded6-44da-81dc-307e15a8480a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:41.864320 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1119/2248] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:27:41 2022] DELETE /identity/v3/projects/0a0c55efa4854edd898655d0d2e5cbd8 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:27:41.868613 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4502a89-28b6-4786-9650-86402cfd545b admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:41.870037 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-d4502a89-28b6-4786-9650-86402cfd545b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:41.887561 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4502a89-28b6-4786-9650-86402cfd545b admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:41.953336 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ec2c1d-4411-4f65-aaa9-8006a6c0f1ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:41.954094 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ec2c1d-4411-4f65-aaa9-8006a6c0f1ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:41.954386 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ec2c1d-4411-4f65-aaa9-8006a6c0f1ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:41.954864 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ec2c1d-4411-4f65-aaa9-8006a6c0f1ad admin admin] PATH_INFO: `/v3/users/48e06cb16565440dac6d090e25cd1802` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:41.956159 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ec2c1d-4411-4f65-aaa9-8006a6c0f1ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=48e06cb16565440dac6d090e25cd1802)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:41.958055 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ec2c1d-4411-4f65-aaa9-8006a6c0f1ad admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:42.161037 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-d7ec2c1d-4411-4f65-aaa9-8006a6c0f1ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48e06cb16565440dac6d090e25cd1802'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:42.161597 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-d7ec2c1d-4411-4f65-aaa9-8006a6c0f1ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48e06cb16565440dac6d090e25cd1802'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:42.173448 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-d7ec2c1d-4411-4f65-aaa9-8006a6c0f1ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48e06cb16565440dac6d090e25cd1802'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:42.243437 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-d7ec2c1d-4411-4f65-aaa9-8006a6c0f1ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48e06cb16565440dac6d090e25cd1802'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:42.244162 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-d7ec2c1d-4411-4f65-aaa9-8006a6c0f1ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48e06cb16565440dac6d090e25cd1802'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:42.258603 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1130/2249] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:27:41 2022] DELETE /identity/v3/users/48e06cb16565440dac6d090e25cd1802 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:27:42.509315 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f275d50f-ded6-44da-81dc-307e15a8480a admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:42.511119 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-f275d50f-ded6-44da-81dc-307e15a8480a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:42.544081 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f275d50f-ded6-44da-81dc-307e15a8480a admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:42.599014 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e952b755-45e5-4747-b361-6c0d7ab83f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:42.599682 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e952b755-45e5-4747-b361-6c0d7ab83f2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:42.599963 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e952b755-45e5-4747-b361-6c0d7ab83f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:42.600228 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e952b755-45e5-4747-b361-6c0d7ab83f2a admin admin] PATH_INFO: `/v3/projects/13f610c86c4541208de543b8cfcf8ede` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:42.601327 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e952b755-45e5-4747-b361-6c0d7ab83f2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=13f610c86c4541208de543b8cfcf8ede)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:42.603213 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e952b755-45e5-4747-b361-6c0d7ab83f2a admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:42.712657 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-e952b755-45e5-4747-b361-6c0d7ab83f2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13f610c86c4541208de543b8cfcf8ede', 'request_id': 'req-e952b755-45e5-4747-b361-6c0d7ab83f2a'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:42.719151 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-e952b755-45e5-4747-b361-6c0d7ab83f2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13f610c86c4541208de543b8cfcf8ede', 'request_id': 'req-e952b755-45e5-4747-b361-6c0d7ab83f2a'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:42.803073 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-e952b755-45e5-4747-b361-6c0d7ab83f2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13f610c86c4541208de543b8cfcf8ede', 'request_id': 'req-e952b755-45e5-4747-b361-6c0d7ab83f2a'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:42.810858 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:27:42.814827 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-e952b755-45e5-4747-b361-6c0d7ab83f2a admin admin] The token cache is being invalidate because project 13f610c86c4541208de543b8cfcf8ede was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81746) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:27:42.815769 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-e952b755-45e5-4747-b361-6c0d7ab83f2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:42.816401 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-e952b755-45e5-4747-b361-6c0d7ab83f2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:42.817891 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1120/2250] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:27:42 2022] DELETE /identity/v3/projects/13f610c86c4541208de543b8cfcf8ede => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:27:42.823087 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79452d2-15d2-4057-8dfc-a86aabae3af1 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:42.823392 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79452d2-15d2-4057-8dfc-a86aabae3af1 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:42.823664 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79452d2-15d2-4057-8dfc-a86aabae3af1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:42.845106 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-a79452d2-15d2-4057-8dfc-a86aabae3af1 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:27:42.846328 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-a79452d2-15d2-4057-8dfc-a86aabae3af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:42.888807 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:27:42.907560 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1131/2251] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:27:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:27:42.915756 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6cc949-96cf-4a09-8a94-ba4655cab9cd None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:42.918409 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6cc949-96cf-4a09-8a94-ba4655cab9cd None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:42.918409 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6cc949-96cf-4a09-8a94-ba4655cab9cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:42.938882 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-6d6cc949-96cf-4a09-8a94-ba4655cab9cd None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:27:42.940659 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-6d6cc949-96cf-4a09-8a94-ba4655cab9cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:42.949231 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1121/2252] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:27:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:27:42.953665 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79452d2-15d2-4057-8dfc-a86aabae3af1 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:42.963710 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79452d2-15d2-4057-8dfc-a86aabae3af1 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:42.968669 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36100fc4-76e2-42c7-b6cd-368650e485e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:42.969269 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36100fc4-76e2-42c7-b6cd-368650e485e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:42.969542 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36100fc4-76e2-42c7-b6cd-368650e485e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:42.969806 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36100fc4-76e2-42c7-b6cd-368650e485e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:42.970441 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36100fc4-76e2-42c7-b6cd-368650e485e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:27:42.970737 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36100fc4-76e2-42c7-b6cd-368650e485e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:42.974337 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36100fc4-76e2-42c7-b6cd-368650e485e8 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:42.984335 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1132/2253] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Oct 12 00:27:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:27:42.989817 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6cc949-96cf-4a09-8a94-ba4655cab9cd None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:42.999939 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6cc949-96cf-4a09-8a94-ba4655cab9cd None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:43.005340 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e98446-57ed-4b06-ae10-44993166ef92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:43.005944 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e98446-57ed-4b06-ae10-44993166ef92 admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:43.006224 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e98446-57ed-4b06-ae10-44993166ef92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:43.006495 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e98446-57ed-4b06-ae10-44993166ef92 admin admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:43.007132 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e98446-57ed-4b06-ae10-44993166ef92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:43.009078 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e98446-57ed-4b06-ae10-44993166ef92 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:43.117708 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1122/2254] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:27:42 2022] POST /identity/v3/projects => generated 373 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:27:43.118858 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36100fc4-76e2-42c7-b6cd-368650e485e8 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:43.129813 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36100fc4-76e2-42c7-b6cd-368650e485e8 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:43.195122 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8536853-e734-45c3-9a3b-4f4144c4b7ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:43.195776 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8536853-e734-45c3-9a3b-4f4144c4b7ac admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:43.196054 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8536853-e734-45c3-9a3b-4f4144c4b7ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:43.196326 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8536853-e734-45c3-9a3b-4f4144c4b7ac admin admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:43.196975 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8536853-e734-45c3-9a3b-4f4144c4b7ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:43.198917 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8536853-e734-45c3-9a3b-4f4144c4b7ac admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:43.324556 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1133/2255] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:27:43 2022] POST /identity/v3/users => generated 287 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 00:27:43.328485 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e98446-57ed-4b06-ae10-44993166ef92 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:43.339140 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e98446-57ed-4b06-ae10-44993166ef92 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:43.343869 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bab88b-5a61-463d-b9ae-ee870cab0d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:43.344524 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bab88b-5a61-463d-b9ae-ee870cab0d54 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:43.344804 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bab88b-5a61-463d-b9ae-ee870cab0d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:43.345077 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bab88b-5a61-463d-b9ae-ee870cab0d54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:43.345722 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bab88b-5a61-463d-b9ae-ee870cab0d54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:43.347865 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bab88b-5a61-463d-b9ae-ee870cab0d54 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:43.357746 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1123/2256] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:27:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:27:43.362083 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8536853-e734-45c3-9a3b-4f4144c4b7ac admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:43.374105 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8536853-e734-45c3-9a3b-4f4144c4b7ac admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:43.379156 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32da664b-2bc9-4a7a-b920-df522b685dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:43.379866 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32da664b-2bc9-4a7a-b920-df522b685dca admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:43.380146 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32da664b-2bc9-4a7a-b920-df522b685dca admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:43.380418 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32da664b-2bc9-4a7a-b920-df522b685dca admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:43.381086 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32da664b-2bc9-4a7a-b920-df522b685dca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:43.382867 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32da664b-2bc9-4a7a-b920-df522b685dca admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:43.394382 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1134/2257] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:27:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:27:43.398572 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bab88b-5a61-463d-b9ae-ee870cab0d54 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:43.408483 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bab88b-5a61-463d-b9ae-ee870cab0d54 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:43.413913 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fb2d42-4092-431e-a6ff-028e622a62b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:43.414597 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb2d42-4092-431e-a6ff-028e622a62b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:43.414894 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb2d42-4092-431e-a6ff-028e622a62b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:43.415190 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb2d42-4092-431e-a6ff-028e622a62b9 admin admin] PATH_INFO: `/v3/projects/92f71b8f614f4282b46d4cc3d03ff385/users/6f3d89766f3a4ac0866dd2b69a029777/roles/ef689f96442945e8a7bc8cbfee13d909` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:43.431777 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fb2d42-4092-431e-a6ff-028e622a62b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92f71b8f614f4282b46d4cc3d03ff385, user_id=6f3d89766f3a4ac0866dd2b69a029777, role_id=ef689f96442945e8a7bc8cbfee13d909)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:43.434449 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fb2d42-4092-431e-a6ff-028e622a62b9 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:43.549088 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1124/2258] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:27:43 2022] PUT /identity/v3/projects/92f71b8f614f4282b46d4cc3d03ff385/users/6f3d89766f3a4ac0866dd2b69a029777/roles/ef689f96442945e8a7bc8cbfee13d909 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:27:43.560658 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a730ec-27ef-4fcf-8c17-dce5bbf1a8a6 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:43.560891 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a730ec-27ef-4fcf-8c17-dce5bbf1a8a6 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:43.561261 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a730ec-27ef-4fcf-8c17-dce5bbf1a8a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:43.593341 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-c6a730ec-27ef-4fcf-8c17-dce5bbf1a8a6 None None] MFA Rules not processed for user `6f3d89766f3a4ac0866dd2b69a029777`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:27:43.594406 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-c6a730ec-27ef-4fcf-8c17-dce5bbf1a8a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:43.666875 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1135/2259] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:27:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:27:43.672193 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fb2d42-4092-431e-a6ff-028e622a62b9 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:43.735001 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fb2d42-4092-431e-a6ff-028e622a62b9 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:43.739802 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783626a9-0da8-4869-a382-9c87ef28bf74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:43.740449 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783626a9-0da8-4869-a382-9c87ef28bf74 admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:43.740727 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783626a9-0da8-4869-a382-9c87ef28bf74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:43.740996 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783626a9-0da8-4869-a382-9c87ef28bf74 admin admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:43.741638 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783626a9-0da8-4869-a382-9c87ef28bf74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:43.743666 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783626a9-0da8-4869-a382-9c87ef28bf74 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:43.834380 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1125/2260] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:27:43 2022] POST /identity/v3/projects => generated 373 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:27:43.838829 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a730ec-27ef-4fcf-8c17-dce5bbf1a8a6 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:43.850931 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a730ec-27ef-4fcf-8c17-dce5bbf1a8a6 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:43.890901 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:27:43.911197 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1852870-888c-44cd-a2df-685aecfc6fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:43.911849 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1852870-888c-44cd-a2df-685aecfc6fe2 admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:43.912131 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1852870-888c-44cd-a2df-685aecfc6fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:43.912405 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1852870-888c-44cd-a2df-685aecfc6fe2 admin admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:43.913040 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1852870-888c-44cd-a2df-685aecfc6fe2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:43.914961 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1852870-888c-44cd-a2df-685aecfc6fe2 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:44.004759 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1136/2261] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:27:43 2022] POST /identity/v3/users => generated 293 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 00:27:44.008361 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783626a9-0da8-4869-a382-9c87ef28bf74 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:44.019511 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783626a9-0da8-4869-a382-9c87ef28bf74 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:44.024107 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267085c5-9020-49f4-b4bc-5dbdbfaa7c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:44.024744 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267085c5-9020-49f4-b4bc-5dbdbfaa7c39 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:44.025040 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267085c5-9020-49f4-b4bc-5dbdbfaa7c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:44.025311 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267085c5-9020-49f4-b4bc-5dbdbfaa7c39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:44.025942 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267085c5-9020-49f4-b4bc-5dbdbfaa7c39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:44.027567 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267085c5-9020-49f4-b4bc-5dbdbfaa7c39 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:44.041280 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1126/2262] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:27:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:27:44.045669 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1852870-888c-44cd-a2df-685aecfc6fe2 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:44.055873 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1852870-888c-44cd-a2df-685aecfc6fe2 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:44.060434 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9282a55e-6aca-4d4b-8473-443c8bf5b70a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:44.061115 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9282a55e-6aca-4d4b-8473-443c8bf5b70a admin admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:44.061390 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9282a55e-6aca-4d4b-8473-443c8bf5b70a admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:44.061664 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9282a55e-6aca-4d4b-8473-443c8bf5b70a admin admin] PATH_INFO: `/v3/projects/1757acd523094d57b418c7a4a9b53227/users/f9374d64878a4ef7abf1267a406837bd/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:44.076834 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9282a55e-6aca-4d4b-8473-443c8bf5b70a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1757acd523094d57b418c7a4a9b53227, user_id=f9374d64878a4ef7abf1267a406837bd, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:44.079522 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9282a55e-6aca-4d4b-8473-443c8bf5b70a admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:44.141745 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1137/2263] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:27:44 2022] PUT /identity/v3/projects/1757acd523094d57b418c7a4a9b53227/users/f9374d64878a4ef7abf1267a406837bd/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:27:44.146986 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267085c5-9020-49f4-b4bc-5dbdbfaa7c39 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:44.157696 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267085c5-9020-49f4-b4bc-5dbdbfaa7c39 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:44.214978 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36d4248-314d-481e-9581-b6a5a1110bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:44.215617 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36d4248-314d-481e-9581-b6a5a1110bc6 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:44.215896 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36d4248-314d-481e-9581-b6a5a1110bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:44.216161 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36d4248-314d-481e-9581-b6a5a1110bc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:44.216797 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36d4248-314d-481e-9581-b6a5a1110bc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:44.218386 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36d4248-314d-481e-9581-b6a5a1110bc6 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:44.227943 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1127/2264] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:27:44 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:27:44.232303 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9282a55e-6aca-4d4b-8473-443c8bf5b70a admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:44.243927 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9282a55e-6aca-4d4b-8473-443c8bf5b70a admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:44.248735 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ece92c1-1f9f-4177-a839-1418afde2090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:44.249390 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ece92c1-1f9f-4177-a839-1418afde2090 admin admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:44.249665 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ece92c1-1f9f-4177-a839-1418afde2090 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:44.249933 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ece92c1-1f9f-4177-a839-1418afde2090 admin admin] PATH_INFO: `/v3/projects/1757acd523094d57b418c7a4a9b53227/users/f9374d64878a4ef7abf1267a406837bd/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:44.251994 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ece92c1-1f9f-4177-a839-1418afde2090 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1757acd523094d57b418c7a4a9b53227, user_id=f9374d64878a4ef7abf1267a406837bd, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:44.255258 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ece92c1-1f9f-4177-a839-1418afde2090 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:44.266567 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1138/2265] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:27:44 2022] PUT /identity/v3/projects/1757acd523094d57b418c7a4a9b53227/users/f9374d64878a4ef7abf1267a406837bd/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:27:44.277594 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35bffb3-6697-44f5-8f7a-8193b566d512 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:44.277878 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35bffb3-6697-44f5-8f7a-8193b566d512 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:44.278156 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35bffb3-6697-44f5-8f7a-8193b566d512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:44.297925 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:27:44.313818 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-b35bffb3-6697-44f5-8f7a-8193b566d512 None None] MFA Rules not processed for user `f9374d64878a4ef7abf1267a406837bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:27:44.315106 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-b35bffb3-6697-44f5-8f7a-8193b566d512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:44.386672 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1128/2266] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:27:44 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:27:44.398236 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ece92c1-1f9f-4177-a839-1418afde2090 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:44.399959 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-6ece92c1-1f9f-4177-a839-1418afde2090 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:44.469196 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ece92c1-1f9f-4177-a839-1418afde2090 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:44.473031 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa2449c-8742-4b5c-b9a5-e1cbd8cdd6e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:44.473638 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa2449c-8742-4b5c-b9a5-e1cbd8cdd6e3 service cinder] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:44.473916 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa2449c-8742-4b5c-b9a5-e1cbd8cdd6e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:44.474184 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa2449c-8742-4b5c-b9a5-e1cbd8cdd6e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:44.483726 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa2449c-8742-4b5c-b9a5-e1cbd8cdd6e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:44.485149 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa2449c-8742-4b5c-b9a5-e1cbd8cdd6e3 service cinder] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:44.495123 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1139/2267] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:27:44 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:27:44.750256 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35bffb3-6697-44f5-8f7a-8193b566d512 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:44.751701 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-b35bffb3-6697-44f5-8f7a-8193b566d512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:44.764075 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35bffb3-6697-44f5-8f7a-8193b566d512 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:44.768368 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b86d786-3e5d-4f69-b075-62d981944bd4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:44.768972 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b86d786-3e5d-4f69-b075-62d981944bd4 service cinder] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:44.769246 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b86d786-3e5d-4f69-b075-62d981944bd4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:44.769509 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b86d786-3e5d-4f69-b075-62d981944bd4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:44.833901 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b86d786-3e5d-4f69-b075-62d981944bd4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:44.835421 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b86d786-3e5d-4f69-b075-62d981944bd4 service cinder] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:44.844438 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1129/2268] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:27:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:27:46.566757 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab55d35-5a87-4299-94da-c34dac8f1ebc None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:46.572061 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab55d35-5a87-4299-94da-c34dac8f1ebc None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:46.572061 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab55d35-5a87-4299-94da-c34dac8f1ebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:46.590644 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-9ab55d35-5a87-4299-94da-c34dac8f1ebc None None] MFA Rules not processed for user `d030d18b595144f19d6ad01d4cd5d52b`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:27:46.591931 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-9ab55d35-5a87-4299-94da-c34dac8f1ebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:46.651488 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1140/2269] 10.222.0.36 () {58 vars in 967 bytes} [Wed Oct 12 00:27:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:27:46.667065 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b86d786-3e5d-4f69-b075-62d981944bd4 service cinder] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:46.669460 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-5b86d786-3e5d-4f69-b075-62d981944bd4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:46.759347 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b86d786-3e5d-4f69-b075-62d981944bd4 service cinder] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:46.769151 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68b9ed0-5e01-461a-baae-08b297ec069a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc1bcffc78844dcbb1a9b7c80285249b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:46.770454 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68b9ed0-5e01-461a-baae-08b297ec069a service swift] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:46.770877 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68b9ed0-5e01-461a-baae-08b297ec069a service swift] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:46.771775 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68b9ed0-5e01-461a-baae-08b297ec069a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:46.780999 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68b9ed0-5e01-461a-baae-08b297ec069a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:46.782782 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68b9ed0-5e01-461a-baae-08b297ec069a service swift] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:46.797563 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1130/2270] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Oct 12 00:27:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 12 00:27:49.338776 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab55d35-5a87-4299-94da-c34dac8f1ebc None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:49.340416 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-9ab55d35-5a87-4299-94da-c34dac8f1ebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:49.385147 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:27:49.408336 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab55d35-5a87-4299-94da-c34dac8f1ebc None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:49.412800 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fd2212-acdd-436d-a6f9-171310230072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:49.413477 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fd2212-acdd-436d-a6f9-171310230072 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:49.413949 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fd2212-acdd-436d-a6f9-171310230072 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:49.414384 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fd2212-acdd-436d-a6f9-171310230072 admin admin] PATH_INFO: `/v3/users/ede8ec1937224e9cae8d619435699ef0` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:49.415890 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fd2212-acdd-436d-a6f9-171310230072 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ede8ec1937224e9cae8d619435699ef0)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:49.417787 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fd2212-acdd-436d-a6f9-171310230072 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:49.617573 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-c3fd2212-acdd-436d-a6f9-171310230072 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ede8ec1937224e9cae8d619435699ef0'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:49.618229 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-c3fd2212-acdd-436d-a6f9-171310230072 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ede8ec1937224e9cae8d619435699ef0'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:49.632672 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-c3fd2212-acdd-436d-a6f9-171310230072 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ede8ec1937224e9cae8d619435699ef0'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:49.688962 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-c3fd2212-acdd-436d-a6f9-171310230072 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ede8ec1937224e9cae8d619435699ef0'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:49.689707 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-c3fd2212-acdd-436d-a6f9-171310230072 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ede8ec1937224e9cae8d619435699ef0'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:49.703280 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1141/2271] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:27:49 2022] DELETE /identity/v3/users/ede8ec1937224e9cae8d619435699ef0 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:27:49.707990 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c962ea51-6ad2-4c34-bf49-666a9e8c90ee None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:49.708449 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c962ea51-6ad2-4c34-bf49-666a9e8c90ee None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:49.708571 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c962ea51-6ad2-4c34-bf49-666a9e8c90ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:49.727477 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-c962ea51-6ad2-4c34-bf49-666a9e8c90ee None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:27:49.728658 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-c962ea51-6ad2-4c34-bf49-666a9e8c90ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:49.741287 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:27:49.789069 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1131/2272] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:27:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:27:49.807383 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fd2212-acdd-436d-a6f9-171310230072 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:49.808790 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-c3fd2212-acdd-436d-a6f9-171310230072 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:49.868452 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fd2212-acdd-436d-a6f9-171310230072 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:49.872007 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b040f3-8992-4881-80f9-785075c47faa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:49.872644 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b040f3-8992-4881-80f9-785075c47faa service neutron] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:49.872917 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b040f3-8992-4881-80f9-785075c47faa service neutron] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:49.873182 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b040f3-8992-4881-80f9-785075c47faa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:49.883990 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b040f3-8992-4881-80f9-785075c47faa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:49.885571 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b040f3-8992-4881-80f9-785075c47faa service neutron] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:49.896185 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1142/2273] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:27:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:27:50.273908 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c962ea51-6ad2-4c34-bf49-666a9e8c90ee None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:50.275334 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-c962ea51-6ad2-4c34-bf49-666a9e8c90ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:50.290513 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c962ea51-6ad2-4c34-bf49-666a9e8c90ee None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:50.295267 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70294cd8-bcd7-44f9-9347-ce81b5fbd515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:50.297166 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70294cd8-bcd7-44f9-9347-ce81b5fbd515 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:50.297480 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70294cd8-bcd7-44f9-9347-ce81b5fbd515 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:50.302184 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70294cd8-bcd7-44f9-9347-ce81b5fbd515 admin admin] PATH_INFO: `/v3/projects/fff997542a6949eeb929bc0bef72c20e` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:50.302184 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70294cd8-bcd7-44f9-9347-ce81b5fbd515 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fff997542a6949eeb929bc0bef72c20e)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:50.302184 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70294cd8-bcd7-44f9-9347-ce81b5fbd515 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:50.419277 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-70294cd8-bcd7-44f9-9347-ce81b5fbd515 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fff997542a6949eeb929bc0bef72c20e', 'request_id': 'req-70294cd8-bcd7-44f9-9347-ce81b5fbd515'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:50.425471 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-70294cd8-bcd7-44f9-9347-ce81b5fbd515 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fff997542a6949eeb929bc0bef72c20e', 'request_id': 'req-70294cd8-bcd7-44f9-9347-ce81b5fbd515'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:50.488386 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-70294cd8-bcd7-44f9-9347-ce81b5fbd515 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fff997542a6949eeb929bc0bef72c20e', 'request_id': 'req-70294cd8-bcd7-44f9-9347-ce81b5fbd515'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:50.498554 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-70294cd8-bcd7-44f9-9347-ce81b5fbd515 admin admin] The token cache is being invalidate because project fff997542a6949eeb929bc0bef72c20e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81746) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:27:50.498838 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-70294cd8-bcd7-44f9-9347-ce81b5fbd515 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:50.499469 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-70294cd8-bcd7-44f9-9347-ce81b5fbd515 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:50.500940 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1132/2274] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:27:50 2022] DELETE /identity/v3/projects/fff997542a6949eeb929bc0bef72c20e => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:27:50.504897 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b040f3-8992-4881-80f9-785075c47faa service neutron] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:50.514277 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b040f3-8992-4881-80f9-785075c47faa service neutron] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:50.570533 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bec7d81-f6d9-4c56-b2ab-ec7fac1dec47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:50.571157 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec7d81-f6d9-4c56-b2ab-ec7fac1dec47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:50.571469 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec7d81-f6d9-4c56-b2ab-ec7fac1dec47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:50.571743 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec7d81-f6d9-4c56-b2ab-ec7fac1dec47 admin admin] PATH_INFO: `/v3/users/8187f9d5817148f29c9a52ed71ee6306` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:50.572859 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bec7d81-f6d9-4c56-b2ab-ec7fac1dec47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8187f9d5817148f29c9a52ed71ee6306)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:50.574759 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bec7d81-f6d9-4c56-b2ab-ec7fac1dec47 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:50.576969 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:27:50.731192 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-6bec7d81-f6d9-4c56-b2ab-ec7fac1dec47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8187f9d5817148f29c9a52ed71ee6306'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:50.731900 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-6bec7d81-f6d9-4c56-b2ab-ec7fac1dec47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8187f9d5817148f29c9a52ed71ee6306'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:50.744554 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-6bec7d81-f6d9-4c56-b2ab-ec7fac1dec47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8187f9d5817148f29c9a52ed71ee6306'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:50.803460 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-6bec7d81-f6d9-4c56-b2ab-ec7fac1dec47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8187f9d5817148f29c9a52ed71ee6306'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:50.804083 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-6bec7d81-f6d9-4c56-b2ab-ec7fac1dec47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8187f9d5817148f29c9a52ed71ee6306'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:50.817385 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1143/2275] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:27:50 2022] DELETE /identity/v3/users/8187f9d5817148f29c9a52ed71ee6306 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:27:51.245190 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70294cd8-bcd7-44f9-9347-ce81b5fbd515 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:51.246639 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-70294cd8-bcd7-44f9-9347-ce81b5fbd515 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:51.259932 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70294cd8-bcd7-44f9-9347-ce81b5fbd515 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:51.321925 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1e0203-db84-43d8-a2d3-dde41eb782c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:51.322559 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e0203-db84-43d8-a2d3-dde41eb782c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:51.322839 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e0203-db84-43d8-a2d3-dde41eb782c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:51.323138 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e0203-db84-43d8-a2d3-dde41eb782c8 admin admin] PATH_INFO: `/v3/projects/852075b062c54374b402e784c2cd19ea` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:51.324308 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1e0203-db84-43d8-a2d3-dde41eb782c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=852075b062c54374b402e784c2cd19ea)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:51.326253 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1e0203-db84-43d8-a2d3-dde41eb782c8 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:51.423058 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:27:51.426594 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-8d1e0203-db84-43d8-a2d3-dde41eb782c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '852075b062c54374b402e784c2cd19ea', 'request_id': 'req-8d1e0203-db84-43d8-a2d3-dde41eb782c8'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:51.432782 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-8d1e0203-db84-43d8-a2d3-dde41eb782c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '852075b062c54374b402e784c2cd19ea', 'request_id': 'req-8d1e0203-db84-43d8-a2d3-dde41eb782c8'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:51.488913 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-8d1e0203-db84-43d8-a2d3-dde41eb782c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '852075b062c54374b402e784c2cd19ea', 'request_id': 'req-8d1e0203-db84-43d8-a2d3-dde41eb782c8'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:51.499329 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-8d1e0203-db84-43d8-a2d3-dde41eb782c8 admin admin] The token cache is being invalidate because project 852075b062c54374b402e784c2cd19ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81746) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:27:51.499788 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-8d1e0203-db84-43d8-a2d3-dde41eb782c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:51.501236 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-8d1e0203-db84-43d8-a2d3-dde41eb782c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:51.502275 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1133/2276] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:27:51 2022] DELETE /identity/v3/projects/852075b062c54374b402e784c2cd19ea => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:27:51.507219 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4a7951-4966-4a85-bfa2-7a4dd9520c75 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:51.507782 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4a7951-4966-4a85-bfa2-7a4dd9520c75 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:51.507877 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4a7951-4966-4a85-bfa2-7a4dd9520c75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:51.527325 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-dc4a7951-4966-4a85-bfa2-7a4dd9520c75 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:27:51.528551 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-dc4a7951-4966-4a85-bfa2-7a4dd9520c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:51.590886 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1144/2277] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:27:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:27:51.600412 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3133647c-bf55-43b7-ac96-d4a59cc56630 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:51.600697 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3133647c-bf55-43b7-ac96-d4a59cc56630 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:51.600966 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3133647c-bf55-43b7-ac96-d4a59cc56630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:51.621254 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-3133647c-bf55-43b7-ac96-d4a59cc56630 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:27:51.622454 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-3133647c-bf55-43b7-ac96-d4a59cc56630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:51.630329 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1134/2278] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:27:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:27:51.634425 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4a7951-4966-4a85-bfa2-7a4dd9520c75 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:51.644333 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4a7951-4966-4a85-bfa2-7a4dd9520c75 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:51.648887 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6004d79-9291-454c-a612-7caa5f8a8df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:51.649480 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6004d79-9291-454c-a612-7caa5f8a8df7 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:51.649755 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6004d79-9291-454c-a612-7caa5f8a8df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:51.650024 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6004d79-9291-454c-a612-7caa5f8a8df7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:51.650660 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6004d79-9291-454c-a612-7caa5f8a8df7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:27:51.650959 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6004d79-9291-454c-a612-7caa5f8a8df7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:51.652586 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6004d79-9291-454c-a612-7caa5f8a8df7 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:51.664378 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1145/2279] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Oct 12 00:27:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 00:27:51.668614 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3133647c-bf55-43b7-ac96-d4a59cc56630 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:51.678281 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3133647c-bf55-43b7-ac96-d4a59cc56630 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:51.682897 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc21a1c1-daba-442a-a0d0-aed19f308b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:51.683526 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc21a1c1-daba-442a-a0d0-aed19f308b00 admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:51.683801 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc21a1c1-daba-442a-a0d0-aed19f308b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:51.684064 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc21a1c1-daba-442a-a0d0-aed19f308b00 admin admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:51.684687 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc21a1c1-daba-442a-a0d0-aed19f308b00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:51.686600 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc21a1c1-daba-442a-a0d0-aed19f308b00 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:51.771518 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1135/2280] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:27:51 2022] POST /identity/v3/projects => generated 387 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:27:51.775723 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6004d79-9291-454c-a612-7caa5f8a8df7 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:51.786296 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6004d79-9291-454c-a612-7caa5f8a8df7 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:51.843465 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df5d83c-cb51-468c-b058-fbf732c2b70f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:51.844078 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df5d83c-cb51-468c-b058-fbf732c2b70f admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:51.844353 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df5d83c-cb51-468c-b058-fbf732c2b70f admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:51.844622 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df5d83c-cb51-468c-b058-fbf732c2b70f admin admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:51.845245 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df5d83c-cb51-468c-b058-fbf732c2b70f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:51.847860 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df5d83c-cb51-468c-b058-fbf732c2b70f admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:51.900209 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:27:51.905982 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1146/2281] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:27:51 2022] POST /identity/v3/users => generated 294 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:27:51.910476 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc21a1c1-daba-442a-a0d0-aed19f308b00 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:51.920803 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc21a1c1-daba-442a-a0d0-aed19f308b00 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:51.925612 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578f68c9-d6e8-46dc-9c64-48b58c019527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:51.930553 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578f68c9-d6e8-46dc-9c64-48b58c019527 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:51.930553 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578f68c9-d6e8-46dc-9c64-48b58c019527 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:51.930553 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578f68c9-d6e8-46dc-9c64-48b58c019527 admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:51.930553 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578f68c9-d6e8-46dc-9c64-48b58c019527 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:51.930553 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578f68c9-d6e8-46dc-9c64-48b58c019527 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:51.939824 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1136/2282] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:27:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:27:51.943783 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df5d83c-cb51-468c-b058-fbf732c2b70f admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:51.954549 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df5d83c-cb51-468c-b058-fbf732c2b70f admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:51.959866 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751d8c36-9e89-4883-a8b5-c7b9162e2d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:51.960585 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751d8c36-9e89-4883-a8b5-c7b9162e2d6b admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:51.960859 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751d8c36-9e89-4883-a8b5-c7b9162e2d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:51.961126 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751d8c36-9e89-4883-a8b5-c7b9162e2d6b admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:51.961757 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751d8c36-9e89-4883-a8b5-c7b9162e2d6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:51.963343 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751d8c36-9e89-4883-a8b5-c7b9162e2d6b admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:51.974540 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1147/2283] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:27:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:27:51.978641 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578f68c9-d6e8-46dc-9c64-48b58c019527 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:51.988528 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578f68c9-d6e8-46dc-9c64-48b58c019527 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:51.993242 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd7f272-725e-42cf-81c4-bf0e15f4f0dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:51.993828 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd7f272-725e-42cf-81c4-bf0e15f4f0dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:51.994102 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd7f272-725e-42cf-81c4-bf0e15f4f0dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:51.994370 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd7f272-725e-42cf-81c4-bf0e15f4f0dd admin admin] PATH_INFO: `/v3/projects/2ff803844eaa4f4a823b794cbf15d675/users/4c109edf559e4beeb12cf3ed085a69d6/roles/ef689f96442945e8a7bc8cbfee13d909` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:52.009251 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd7f272-725e-42cf-81c4-bf0e15f4f0dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ff803844eaa4f4a823b794cbf15d675, user_id=4c109edf559e4beeb12cf3ed085a69d6, role_id=ef689f96442945e8a7bc8cbfee13d909)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:52.012004 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd7f272-725e-42cf-81c4-bf0e15f4f0dd admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:52.064932 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1137/2284] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:27:51 2022] PUT /identity/v3/projects/2ff803844eaa4f4a823b794cbf15d675/users/4c109edf559e4beeb12cf3ed085a69d6/roles/ef689f96442945e8a7bc8cbfee13d909 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:27:52.075263 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c23fd05-a6dc-4c1a-9ab2-a0dc1985b873 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:52.075591 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c23fd05-a6dc-4c1a-9ab2-a0dc1985b873 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:52.075864 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c23fd05-a6dc-4c1a-9ab2-a0dc1985b873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:52.105681 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-1c23fd05-a6dc-4c1a-9ab2-a0dc1985b873 None None] MFA Rules not processed for user `4c109edf559e4beeb12cf3ed085a69d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:27:52.106892 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-1c23fd05-a6dc-4c1a-9ab2-a0dc1985b873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:52.170016 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1148/2285] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:27:52 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:27:52.174652 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd7f272-725e-42cf-81c4-bf0e15f4f0dd admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:52.237684 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd7f272-725e-42cf-81c4-bf0e15f4f0dd admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:52.242428 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a2bf3c-6392-4a8a-92ce-4f0f21dc37e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:52.243033 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a2bf3c-6392-4a8a-92ce-4f0f21dc37e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:52.243465 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a2bf3c-6392-4a8a-92ce-4f0f21dc37e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:52.243650 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a2bf3c-6392-4a8a-92ce-4f0f21dc37e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:52.244305 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a2bf3c-6392-4a8a-92ce-4f0f21dc37e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:52.246228 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a2bf3c-6392-4a8a-92ce-4f0f21dc37e0 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:52.330396 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1138/2286] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:27:52 2022] POST /identity/v3/projects => generated 387 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:27:52.334314 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c23fd05-a6dc-4c1a-9ab2-a0dc1985b873 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:52.345382 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c23fd05-a6dc-4c1a-9ab2-a0dc1985b873 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:52.401982 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b8be33-6934-4252-8eea-d2b324e2b65b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:52.402586 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b8be33-6934-4252-8eea-d2b324e2b65b admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:52.402862 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b8be33-6934-4252-8eea-d2b324e2b65b admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:52.403131 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b8be33-6934-4252-8eea-d2b324e2b65b admin admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:52.403791 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b8be33-6934-4252-8eea-d2b324e2b65b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:52.405698 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b8be33-6934-4252-8eea-d2b324e2b65b admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:52.472697 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1149/2287] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:27:52 2022] POST /identity/v3/users => generated 300 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:27:52.476666 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a2bf3c-6392-4a8a-92ce-4f0f21dc37e0 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:52.486919 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a2bf3c-6392-4a8a-92ce-4f0f21dc37e0 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:52.491627 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b751e8-b665-4e0f-aa9b-99a46069b1a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:52.492472 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b751e8-b665-4e0f-aa9b-99a46069b1a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:52.492597 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b751e8-b665-4e0f-aa9b-99a46069b1a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:52.492786 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b751e8-b665-4e0f-aa9b-99a46069b1a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:52.493421 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b751e8-b665-4e0f-aa9b-99a46069b1a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:52.495011 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b751e8-b665-4e0f-aa9b-99a46069b1a5 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:52.505646 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1139/2288] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:27:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:27:52.509770 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b8be33-6934-4252-8eea-d2b324e2b65b admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:52.520471 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b8be33-6934-4252-8eea-d2b324e2b65b admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:52.525002 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1da9bf-b4a9-46b7-ae8a-df18c8e8025a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:52.525639 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1da9bf-b4a9-46b7-ae8a-df18c8e8025a admin admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:52.525911 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1da9bf-b4a9-46b7-ae8a-df18c8e8025a admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:52.526213 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1da9bf-b4a9-46b7-ae8a-df18c8e8025a admin admin] PATH_INFO: `/v3/projects/8c15b0c0047e4ba0b578f88cfde94b36/users/1b07c9330fbc43678938d8ec494d8a07/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:52.541685 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1da9bf-b4a9-46b7-ae8a-df18c8e8025a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c15b0c0047e4ba0b578f88cfde94b36, user_id=1b07c9330fbc43678938d8ec494d8a07, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:52.544411 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1da9bf-b4a9-46b7-ae8a-df18c8e8025a admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:52.620256 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1150/2289] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:27:52 2022] PUT /identity/v3/projects/8c15b0c0047e4ba0b578f88cfde94b36/users/1b07c9330fbc43678938d8ec494d8a07/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:27:52.624245 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b751e8-b665-4e0f-aa9b-99a46069b1a5 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:52.634386 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b751e8-b665-4e0f-aa9b-99a46069b1a5 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:52.676083 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:27:52.690339 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab90ee1d-f3fb-4540-b119-4d829ca6f2c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:52.690937 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab90ee1d-f3fb-4540-b119-4d829ca6f2c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:52.691212 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab90ee1d-f3fb-4540-b119-4d829ca6f2c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:52.691589 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab90ee1d-f3fb-4540-b119-4d829ca6f2c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:52.692227 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab90ee1d-f3fb-4540-b119-4d829ca6f2c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:52.693828 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab90ee1d-f3fb-4540-b119-4d829ca6f2c0 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:52.703895 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1140/2290] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:27:52 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:27:52.707951 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1da9bf-b4a9-46b7-ae8a-df18c8e8025a admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:52.718393 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1da9bf-b4a9-46b7-ae8a-df18c8e8025a admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:52.723997 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b488e8-a02d-4779-9ea6-c15d4701de9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:52.724640 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b488e8-a02d-4779-9ea6-c15d4701de9f admin admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:52.724914 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b488e8-a02d-4779-9ea6-c15d4701de9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:52.725183 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b488e8-a02d-4779-9ea6-c15d4701de9f admin admin] PATH_INFO: `/v3/projects/8c15b0c0047e4ba0b578f88cfde94b36/users/1b07c9330fbc43678938d8ec494d8a07/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:52.727080 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b488e8-a02d-4779-9ea6-c15d4701de9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c15b0c0047e4ba0b578f88cfde94b36, user_id=1b07c9330fbc43678938d8ec494d8a07, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:52.729749 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b488e8-a02d-4779-9ea6-c15d4701de9f admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:52.739872 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1151/2291] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:27:52 2022] PUT /identity/v3/projects/8c15b0c0047e4ba0b578f88cfde94b36/users/1b07c9330fbc43678938d8ec494d8a07/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:27:52.749912 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caecb12-e4bd-439c-9e95-c156d77182dd None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:52.750138 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caecb12-e4bd-439c-9e95-c156d77182dd None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:52.750412 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caecb12-e4bd-439c-9e95-c156d77182dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:52.778165 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-4caecb12-e4bd-439c-9e95-c156d77182dd None None] MFA Rules not processed for user `1b07c9330fbc43678938d8ec494d8a07`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:27:52.779359 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-4caecb12-e4bd-439c-9e95-c156d77182dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:52.848136 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1141/2292] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:27:52 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:27:52.859915 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b488e8-a02d-4779-9ea6-c15d4701de9f admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:52.861622 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-b0b488e8-a02d-4779-9ea6-c15d4701de9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:52.896075 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:27:52.921405 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b488e8-a02d-4779-9ea6-c15d4701de9f admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:52.925291 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7ffb50-5b9a-4eb3-993d-5861ad9c7191 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:52.925905 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7ffb50-5b9a-4eb3-993d-5861ad9c7191 service cinder] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:52.926216 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7ffb50-5b9a-4eb3-993d-5861ad9c7191 service cinder] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:52.926499 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7ffb50-5b9a-4eb3-993d-5861ad9c7191 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:52.936354 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7ffb50-5b9a-4eb3-993d-5861ad9c7191 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:52.940091 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7ffb50-5b9a-4eb3-993d-5861ad9c7191 service cinder] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:52.948023 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1152/2293] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:27:52 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:27:53.029040 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caecb12-e4bd-439c-9e95-c156d77182dd None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:53.099921 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caecb12-e4bd-439c-9e95-c156d77182dd None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:53.104650 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dc5085-d9fd-4622-a422-b542c2f9e4ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:53.105343 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dc5085-d9fd-4622-a422-b542c2f9e4ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:53.105626 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dc5085-d9fd-4622-a422-b542c2f9e4ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:53.105898 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dc5085-d9fd-4622-a422-b542c2f9e4ef admin admin] PATH_INFO: `/v3/users/4c109edf559e4beeb12cf3ed085a69d6` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:53.107147 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dc5085-d9fd-4622-a422-b542c2f9e4ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c109edf559e4beeb12cf3ed085a69d6)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:53.109182 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dc5085-d9fd-4622-a422-b542c2f9e4ef admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:53.281066 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-d1dc5085-d9fd-4622-a422-b542c2f9e4ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c109edf559e4beeb12cf3ed085a69d6'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:53.281804 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-d1dc5085-d9fd-4622-a422-b542c2f9e4ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c109edf559e4beeb12cf3ed085a69d6'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:53.294697 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-d1dc5085-d9fd-4622-a422-b542c2f9e4ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c109edf559e4beeb12cf3ed085a69d6'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:53.297437 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:27:53.334598 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-d1dc5085-d9fd-4622-a422-b542c2f9e4ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c109edf559e4beeb12cf3ed085a69d6'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:53.335222 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-d1dc5085-d9fd-4622-a422-b542c2f9e4ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c109edf559e4beeb12cf3ed085a69d6'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:53.348753 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1142/2294] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:27:53 2022] DELETE /identity/v3/users/4c109edf559e4beeb12cf3ed085a69d6 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:27:53.353409 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348da410-7414-44ab-addb-4456d4891694 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:53.353780 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348da410-7414-44ab-addb-4456d4891694 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:53.354062 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348da410-7414-44ab-addb-4456d4891694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:53.373804 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-348da410-7414-44ab-addb-4456d4891694 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:27:53.375055 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-348da410-7414-44ab-addb-4456d4891694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:53.433890 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1153/2295] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:27:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:27:53.445554 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dc5085-d9fd-4622-a422-b542c2f9e4ef admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:53.447048 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-d1dc5085-d9fd-4622-a422-b542c2f9e4ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:53.504659 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dc5085-d9fd-4622-a422-b542c2f9e4ef admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:53.508346 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d5a881-340a-48dd-a263-8f9af5c5834a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:53.508948 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d5a881-340a-48dd-a263-8f9af5c5834a service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:53.509226 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d5a881-340a-48dd-a263-8f9af5c5834a service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:53.509494 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d5a881-340a-48dd-a263-8f9af5c5834a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:53.518652 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d5a881-340a-48dd-a263-8f9af5c5834a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:53.520115 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d5a881-340a-48dd-a263-8f9af5c5834a service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:53.531867 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1143/2296] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:27:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:27:53.947563 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348da410-7414-44ab-addb-4456d4891694 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:53.949090 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-348da410-7414-44ab-addb-4456d4891694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:53.961918 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348da410-7414-44ab-addb-4456d4891694 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:53.968088 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed52adcd-2168-4f37-9e6e-b057d8326d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:53.968088 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed52adcd-2168-4f37-9e6e-b057d8326d26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:53.968414 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed52adcd-2168-4f37-9e6e-b057d8326d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:53.968586 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed52adcd-2168-4f37-9e6e-b057d8326d26 admin admin] PATH_INFO: `/v3/projects/2ff803844eaa4f4a823b794cbf15d675` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:53.969696 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed52adcd-2168-4f37-9e6e-b057d8326d26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ff803844eaa4f4a823b794cbf15d675)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:53.971642 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed52adcd-2168-4f37-9e6e-b057d8326d26 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:54.054078 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-ed52adcd-2168-4f37-9e6e-b057d8326d26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ff803844eaa4f4a823b794cbf15d675', 'request_id': 'req-ed52adcd-2168-4f37-9e6e-b057d8326d26'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:54.060328 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-ed52adcd-2168-4f37-9e6e-b057d8326d26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ff803844eaa4f4a823b794cbf15d675', 'request_id': 'req-ed52adcd-2168-4f37-9e6e-b057d8326d26'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:54.145769 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-ed52adcd-2168-4f37-9e6e-b057d8326d26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ff803844eaa4f4a823b794cbf15d675', 'request_id': 'req-ed52adcd-2168-4f37-9e6e-b057d8326d26'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:54.158241 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-ed52adcd-2168-4f37-9e6e-b057d8326d26 admin admin] The token cache is being invalidate because project 2ff803844eaa4f4a823b794cbf15d675 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81747) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:27:54.158241 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-ed52adcd-2168-4f37-9e6e-b057d8326d26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:54.158241 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-ed52adcd-2168-4f37-9e6e-b057d8326d26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:54.160174 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1154/2297] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:27:53 2022] DELETE /identity/v3/projects/2ff803844eaa4f4a823b794cbf15d675 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:27:54.164177 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d5a881-340a-48dd-a263-8f9af5c5834a service neutron] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:54.173979 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d5a881-340a-48dd-a263-8f9af5c5834a service neutron] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:54.230439 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db718eb-c605-4157-84da-0be396dfee17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:54.231063 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db718eb-c605-4157-84da-0be396dfee17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:54.231339 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db718eb-c605-4157-84da-0be396dfee17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:54.231638 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db718eb-c605-4157-84da-0be396dfee17 admin admin] PATH_INFO: `/v3/users/1b07c9330fbc43678938d8ec494d8a07` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:54.232746 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db718eb-c605-4157-84da-0be396dfee17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b07c9330fbc43678938d8ec494d8a07)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:54.234699 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db718eb-c605-4157-84da-0be396dfee17 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:54.409488 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-0db718eb-c605-4157-84da-0be396dfee17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b07c9330fbc43678938d8ec494d8a07'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:54.410180 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-0db718eb-c605-4157-84da-0be396dfee17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b07c9330fbc43678938d8ec494d8a07'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:54.421834 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-0db718eb-c605-4157-84da-0be396dfee17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b07c9330fbc43678938d8ec494d8a07'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:54.476941 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-0db718eb-c605-4157-84da-0be396dfee17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b07c9330fbc43678938d8ec494d8a07'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:54.477537 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-0db718eb-c605-4157-84da-0be396dfee17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b07c9330fbc43678938d8ec494d8a07'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:54.490094 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1144/2298] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:27:54 2022] DELETE /identity/v3/users/1b07c9330fbc43678938d8ec494d8a07 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:27:54.873179 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed52adcd-2168-4f37-9e6e-b057d8326d26 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:54.874659 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-ed52adcd-2168-4f37-9e6e-b057d8326d26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:54.888587 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed52adcd-2168-4f37-9e6e-b057d8326d26 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:54.940507 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:27:54.948922 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3217196-adf8-4dfe-b3ad-a13daa90f534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:54.950415 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3217196-adf8-4dfe-b3ad-a13daa90f534 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:54.950415 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3217196-adf8-4dfe-b3ad-a13daa90f534 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:54.950415 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3217196-adf8-4dfe-b3ad-a13daa90f534 admin admin] PATH_INFO: `/v3/projects/8c15b0c0047e4ba0b578f88cfde94b36` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:54.951318 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3217196-adf8-4dfe-b3ad-a13daa90f534 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c15b0c0047e4ba0b578f88cfde94b36)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:54.953260 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3217196-adf8-4dfe-b3ad-a13daa90f534 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:55.068806 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-f3217196-adf8-4dfe-b3ad-a13daa90f534 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c15b0c0047e4ba0b578f88cfde94b36', 'request_id': 'req-f3217196-adf8-4dfe-b3ad-a13daa90f534'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:55.075301 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-f3217196-adf8-4dfe-b3ad-a13daa90f534 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c15b0c0047e4ba0b578f88cfde94b36', 'request_id': 'req-f3217196-adf8-4dfe-b3ad-a13daa90f534'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:55.133559 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-f3217196-adf8-4dfe-b3ad-a13daa90f534 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c15b0c0047e4ba0b578f88cfde94b36', 'request_id': 'req-f3217196-adf8-4dfe-b3ad-a13daa90f534'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:55.147014 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-f3217196-adf8-4dfe-b3ad-a13daa90f534 admin admin] The token cache is being invalidate because project 8c15b0c0047e4ba0b578f88cfde94b36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81747) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:27:55.147225 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-f3217196-adf8-4dfe-b3ad-a13daa90f534 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:55.147914 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-f3217196-adf8-4dfe-b3ad-a13daa90f534 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:27:55.149410 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1155/2299] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:27:54 2022] DELETE /identity/v3/projects/8c15b0c0047e4ba0b578f88cfde94b36 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:27:55.154542 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4774a1c9-8768-4b6f-bbcf-85f2e9b9294f None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:55.154851 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4774a1c9-8768-4b6f-bbcf-85f2e9b9294f None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:55.155130 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4774a1c9-8768-4b6f-bbcf-85f2e9b9294f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:55.170048 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:27:55.176800 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-4774a1c9-8768-4b6f-bbcf-85f2e9b9294f None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:27:55.177985 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-4774a1c9-8768-4b6f-bbcf-85f2e9b9294f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:55.242325 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1145/2300] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:27:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:27:55.252464 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424d42ae-6441-49eb-9839-7f67ee36c0e9 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:55.252829 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424d42ae-6441-49eb-9839-7f67ee36c0e9 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:55.253101 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424d42ae-6441-49eb-9839-7f67ee36c0e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:55.273989 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-424d42ae-6441-49eb-9839-7f67ee36c0e9 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:27:55.275190 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-424d42ae-6441-49eb-9839-7f67ee36c0e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:55.283075 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1156/2301] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:27:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:27:55.288031 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4774a1c9-8768-4b6f-bbcf-85f2e9b9294f None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:55.297719 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4774a1c9-8768-4b6f-bbcf-85f2e9b9294f None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:55.303010 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e7b77-b99f-4725-a584-273c64a81894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:55.303644 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e7b77-b99f-4725-a584-273c64a81894 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:55.303926 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e7b77-b99f-4725-a584-273c64a81894 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:55.304198 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e7b77-b99f-4725-a584-273c64a81894 admin admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:55.304849 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767e7b77-b99f-4725-a584-273c64a81894 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:27:55.305143 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767e7b77-b99f-4725-a584-273c64a81894 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:55.306733 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767e7b77-b99f-4725-a584-273c64a81894 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:55.318186 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1146/2302] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Oct 12 00:27:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:27:55.326835 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424d42ae-6441-49eb-9839-7f67ee36c0e9 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:55.336258 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424d42ae-6441-49eb-9839-7f67ee36c0e9 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:55.340965 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e4a472-4031-4f9d-87dd-59a393b523a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:55.341561 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e4a472-4031-4f9d-87dd-59a393b523a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:55.341835 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e4a472-4031-4f9d-87dd-59a393b523a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:55.342102 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e4a472-4031-4f9d-87dd-59a393b523a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:55.342728 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e4a472-4031-4f9d-87dd-59a393b523a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:55.344814 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e4a472-4031-4f9d-87dd-59a393b523a5 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:55.441520 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1157/2303] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:27:55 2022] POST /identity/v3/projects => generated 401 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 00:27:55.442643 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e7b77-b99f-4725-a584-273c64a81894 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:55.452835 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e7b77-b99f-4725-a584-273c64a81894 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:55.512102 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828cca4a-39e9-4928-b7ad-36a0349dfe59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:55.512706 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828cca4a-39e9-4928-b7ad-36a0349dfe59 admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:55.512980 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828cca4a-39e9-4928-b7ad-36a0349dfe59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:55.513247 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828cca4a-39e9-4928-b7ad-36a0349dfe59 admin admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:55.513859 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828cca4a-39e9-4928-b7ad-36a0349dfe59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:55.515861 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828cca4a-39e9-4928-b7ad-36a0349dfe59 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:55.614909 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1147/2304] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:27:55 2022] POST /identity/v3/users => generated 301 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:27:55.619450 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e4a472-4031-4f9d-87dd-59a393b523a5 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:55.630032 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e4a472-4031-4f9d-87dd-59a393b523a5 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:55.634724 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80be4cb3-4a28-4467-9b23-0332a33722de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:55.635324 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be4cb3-4a28-4467-9b23-0332a33722de admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:55.635661 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be4cb3-4a28-4467-9b23-0332a33722de admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:55.635935 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be4cb3-4a28-4467-9b23-0332a33722de admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:55.636570 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80be4cb3-4a28-4467-9b23-0332a33722de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:55.638157 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80be4cb3-4a28-4467-9b23-0332a33722de admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:55.648802 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1158/2305] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:27:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:27:55.656093 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828cca4a-39e9-4928-b7ad-36a0349dfe59 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:55.663521 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828cca4a-39e9-4928-b7ad-36a0349dfe59 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:55.668133 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7688f90-0799-4d0a-90f6-24cc7f18b06b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:55.668726 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7688f90-0799-4d0a-90f6-24cc7f18b06b admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:55.669005 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7688f90-0799-4d0a-90f6-24cc7f18b06b admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:55.669279 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7688f90-0799-4d0a-90f6-24cc7f18b06b admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:55.670089 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7688f90-0799-4d0a-90f6-24cc7f18b06b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:55.671718 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7688f90-0799-4d0a-90f6-24cc7f18b06b admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:55.682252 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1148/2306] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:27:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:27:55.686184 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80be4cb3-4a28-4467-9b23-0332a33722de admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:55.696468 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80be4cb3-4a28-4467-9b23-0332a33722de admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:55.701162 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf26b06-c0fa-403e-b0d6-c0329f02044e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:55.701798 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf26b06-c0fa-403e-b0d6-c0329f02044e admin admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:55.702071 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf26b06-c0fa-403e-b0d6-c0329f02044e admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:55.702337 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf26b06-c0fa-403e-b0d6-c0329f02044e admin admin] PATH_INFO: `/v3/projects/0b3b7ef5ca3c4d72b9f1ae325f0970ef/users/1a4fdac626db48718106d6cd987d51c9/roles/ef689f96442945e8a7bc8cbfee13d909` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:55.717185 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf26b06-c0fa-403e-b0d6-c0329f02044e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b3b7ef5ca3c4d72b9f1ae325f0970ef, user_id=1a4fdac626db48718106d6cd987d51c9, role_id=ef689f96442945e8a7bc8cbfee13d909)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:55.719913 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf26b06-c0fa-403e-b0d6-c0329f02044e admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:55.814641 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1159/2307] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:27:55 2022] PUT /identity/v3/projects/0b3b7ef5ca3c4d72b9f1ae325f0970ef/users/1a4fdac626db48718106d6cd987d51c9/roles/ef689f96442945e8a7bc8cbfee13d909 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 00:27:55.825443 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd19b92f-e92b-4028-a755-47095ec1cae0 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:55.825729 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd19b92f-e92b-4028-a755-47095ec1cae0 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:55.825999 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd19b92f-e92b-4028-a755-47095ec1cae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:55.853644 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-fd19b92f-e92b-4028-a755-47095ec1cae0 None None] MFA Rules not processed for user `1a4fdac626db48718106d6cd987d51c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:27:55.854825 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-fd19b92f-e92b-4028-a755-47095ec1cae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:55.917261 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1149/2308] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:27:55 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:27:55.921768 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf26b06-c0fa-403e-b0d6-c0329f02044e admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:55.982886 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf26b06-c0fa-403e-b0d6-c0329f02044e admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:55.987475 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed351f5-f22f-46c5-bc79-0b35498c5404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:55.988076 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed351f5-f22f-46c5-bc79-0b35498c5404 admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:55.988348 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed351f5-f22f-46c5-bc79-0b35498c5404 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:55.988615 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed351f5-f22f-46c5-bc79-0b35498c5404 admin admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:55.989234 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed351f5-f22f-46c5-bc79-0b35498c5404 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:55.991128 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed351f5-f22f-46c5-bc79-0b35498c5404 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:56.111831 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1160/2309] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:27:55 2022] POST /identity/v3/projects => generated 401 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:27:56.115912 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd19b92f-e92b-4028-a755-47095ec1cae0 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:56.121290 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:27:56.126951 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd19b92f-e92b-4028-a755-47095ec1cae0 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:56.182646 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede002df-9777-401f-bddc-6909a5595599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:56.183248 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede002df-9777-401f-bddc-6909a5595599 admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:56.183557 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede002df-9777-401f-bddc-6909a5595599 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:56.183832 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede002df-9777-401f-bddc-6909a5595599 admin admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:56.184460 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede002df-9777-401f-bddc-6909a5595599 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:56.186391 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede002df-9777-401f-bddc-6909a5595599 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:56.305698 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1150/2310] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:27:56 2022] POST /identity/v3/users => generated 307 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:27:56.309740 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed351f5-f22f-46c5-bc79-0b35498c5404 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:56.320620 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed351f5-f22f-46c5-bc79-0b35498c5404 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:56.325193 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4643ba0-628e-4efb-b91b-c542a90314ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:56.325817 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4643ba0-628e-4efb-b91b-c542a90314ac admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:56.326092 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4643ba0-628e-4efb-b91b-c542a90314ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:56.326364 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4643ba0-628e-4efb-b91b-c542a90314ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:56.327034 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4643ba0-628e-4efb-b91b-c542a90314ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:56.328653 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4643ba0-628e-4efb-b91b-c542a90314ac admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:56.329922 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:27:56.339559 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1161/2311] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:27:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:27:56.347402 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede002df-9777-401f-bddc-6909a5595599 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:56.357274 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede002df-9777-401f-bddc-6909a5595599 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:56.362064 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220d7c5f-3dc4-4473-a237-9b4073c641ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:56.362710 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220d7c5f-3dc4-4473-a237-9b4073c641ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:56.363013 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220d7c5f-3dc4-4473-a237-9b4073c641ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:56.363793 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220d7c5f-3dc4-4473-a237-9b4073c641ce admin admin] PATH_INFO: `/v3/projects/34c86a4c72c74c9aa1981aa955764416/users/dd13da8b7c3b437e875495369a6f7287/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:56.378470 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220d7c5f-3dc4-4473-a237-9b4073c641ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=34c86a4c72c74c9aa1981aa955764416, user_id=dd13da8b7c3b437e875495369a6f7287, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:56.381788 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220d7c5f-3dc4-4473-a237-9b4073c641ce admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:56.449119 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1151/2312] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:27:56 2022] PUT /identity/v3/projects/34c86a4c72c74c9aa1981aa955764416/users/dd13da8b7c3b437e875495369a6f7287/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 00:27:56.453186 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4643ba0-628e-4efb-b91b-c542a90314ac admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:56.463001 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4643ba0-628e-4efb-b91b-c542a90314ac admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:56.519839 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f03e7e-78bd-4db5-a663-e4feee65c6e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:56.520606 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f03e7e-78bd-4db5-a663-e4feee65c6e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:56.520884 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f03e7e-78bd-4db5-a663-e4feee65c6e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:56.521156 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f03e7e-78bd-4db5-a663-e4feee65c6e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:56.521910 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f03e7e-78bd-4db5-a663-e4feee65c6e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:56.523553 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f03e7e-78bd-4db5-a663-e4feee65c6e9 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:56.534098 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1162/2313] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:27:56 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:27:56.538090 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220d7c5f-3dc4-4473-a237-9b4073c641ce admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:56.551026 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220d7c5f-3dc4-4473-a237-9b4073c641ce admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:56.563847 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0faeb0-9209-4d5f-81d0-cd24c572f6c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:56.563847 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0faeb0-9209-4d5f-81d0-cd24c572f6c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:56.563847 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0faeb0-9209-4d5f-81d0-cd24c572f6c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:56.563847 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0faeb0-9209-4d5f-81d0-cd24c572f6c8 admin admin] PATH_INFO: `/v3/projects/34c86a4c72c74c9aa1981aa955764416/users/dd13da8b7c3b437e875495369a6f7287/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:56.564164 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0faeb0-9209-4d5f-81d0-cd24c572f6c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34c86a4c72c74c9aa1981aa955764416, user_id=dd13da8b7c3b437e875495369a6f7287, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:56.564594 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0faeb0-9209-4d5f-81d0-cd24c572f6c8 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:56.576674 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1152/2314] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:27:56 2022] PUT /identity/v3/projects/34c86a4c72c74c9aa1981aa955764416/users/dd13da8b7c3b437e875495369a6f7287/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:27:56.586500 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aaa867-633e-4f23-a508-d9479972f27b None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:56.586775 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aaa867-633e-4f23-a508-d9479972f27b None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:56.587049 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aaa867-633e-4f23-a508-d9479972f27b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:56.616900 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-b7aaa867-633e-4f23-a508-d9479972f27b None None] MFA Rules not processed for user `dd13da8b7c3b437e875495369a6f7287`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:27:56.618112 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-b7aaa867-633e-4f23-a508-d9479972f27b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:56.687069 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1163/2315] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:27:56 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:27:56.698276 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0faeb0-9209-4d5f-81d0-cd24c572f6c8 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:56.700086 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-ef0faeb0-9209-4d5f-81d0-cd24c572f6c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:56.757797 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0faeb0-9209-4d5f-81d0-cd24c572f6c8 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:56.761515 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16c9c3f-a4d5-46a8-b4e7-7c799261f8e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:56.762120 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16c9c3f-a4d5-46a8-b4e7-7c799261f8e9 service cinder] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:56.762397 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16c9c3f-a4d5-46a8-b4e7-7c799261f8e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:56.762666 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16c9c3f-a4d5-46a8-b4e7-7c799261f8e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:56.777705 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16c9c3f-a4d5-46a8-b4e7-7c799261f8e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:56.779151 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16c9c3f-a4d5-46a8-b4e7-7c799261f8e9 service cinder] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:56.789622 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1153/2316] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:27:56 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:27:56.990635 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aaa867-633e-4f23-a508-d9479972f27b None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:27:56.992056 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-b7aaa867-633e-4f23-a508-d9479972f27b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:27:57.003756 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aaa867-633e-4f23-a508-d9479972f27b None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:27:57.007309 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8300e5f-b103-435e-bc22-e4b3a2246fb0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:27:57.007893 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8300e5f-b103-435e-bc22-e4b3a2246fb0 service cinder] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:27:57.008161 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8300e5f-b103-435e-bc22-e4b3a2246fb0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:27:57.008426 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8300e5f-b103-435e-bc22-e4b3a2246fb0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:27:57.067925 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8300e5f-b103-435e-bc22-e4b3a2246fb0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:27:57.069345 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8300e5f-b103-435e-bc22-e4b3a2246fb0 service cinder] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:27:57.078303 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1164/2317] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:27:56 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:28:02.377051 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16c9c3f-a4d5-46a8-b4e7-7c799261f8e9 service cinder] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:02.440390 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:28:02.445335 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16c9c3f-a4d5-46a8-b4e7-7c799261f8e9 service cinder] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:02.450018 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9ce574-16e9-4a5e-afdb-24d6f6f4ed74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:02.450694 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9ce574-16e9-4a5e-afdb-24d6f6f4ed74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:02.451003 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9ce574-16e9-4a5e-afdb-24d6f6f4ed74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:02.451274 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9ce574-16e9-4a5e-afdb-24d6f6f4ed74 admin admin] PATH_INFO: `/v3/users/1a4fdac626db48718106d6cd987d51c9` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:02.452547 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9ce574-16e9-4a5e-afdb-24d6f6f4ed74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a4fdac626db48718106d6cd987d51c9)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:02.454489 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9ce574-16e9-4a5e-afdb-24d6f6f4ed74 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:02.782999 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-6b9ce574-16e9-4a5e-afdb-24d6f6f4ed74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a4fdac626db48718106d6cd987d51c9'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:02.783707 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-6b9ce574-16e9-4a5e-afdb-24d6f6f4ed74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a4fdac626db48718106d6cd987d51c9'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:02.796493 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-6b9ce574-16e9-4a5e-afdb-24d6f6f4ed74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a4fdac626db48718106d6cd987d51c9'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:02.854423 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-6b9ce574-16e9-4a5e-afdb-24d6f6f4ed74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a4fdac626db48718106d6cd987d51c9'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:02.855046 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-6b9ce574-16e9-4a5e-afdb-24d6f6f4ed74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a4fdac626db48718106d6cd987d51c9'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:02.869184 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1154/2318] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:28:02 2022] DELETE /identity/v3/users/1a4fdac626db48718106d6cd987d51c9 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:28:02.873900 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a899d83-1d5e-4d5d-ab45-ccbdbe25b654 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:02.874379 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a899d83-1d5e-4d5d-ab45-ccbdbe25b654 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:02.874495 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a899d83-1d5e-4d5d-ab45-ccbdbe25b654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:02.894113 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-7a899d83-1d5e-4d5d-ab45-ccbdbe25b654 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:28:02.895335 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-7a899d83-1d5e-4d5d-ab45-ccbdbe25b654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:28:02.906032 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:28:02.956315 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1165/2319] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:28:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 00:28:02.973550 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9ce574-16e9-4a5e-afdb-24d6f6f4ed74 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:02.975633 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-6b9ce574-16e9-4a5e-afdb-24d6f6f4ed74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:28:03.040083 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9ce574-16e9-4a5e-afdb-24d6f6f4ed74 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:03.044050 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef147074-80ca-4ec7-88e7-40ca4a2b7752 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:03.044983 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef147074-80ca-4ec7-88e7-40ca4a2b7752 service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:03.044983 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef147074-80ca-4ec7-88e7-40ca4a2b7752 service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:03.045278 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef147074-80ca-4ec7-88e7-40ca4a2b7752 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:03.059416 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef147074-80ca-4ec7-88e7-40ca4a2b7752 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:03.066329 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef147074-80ca-4ec7-88e7-40ca4a2b7752 service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:03.077289 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1155/2320] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:28:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:28:03.564274 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a899d83-1d5e-4d5d-ab45-ccbdbe25b654 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:03.564274 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-7a899d83-1d5e-4d5d-ab45-ccbdbe25b654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:28:03.573682 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a899d83-1d5e-4d5d-ab45-ccbdbe25b654 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:03.583294 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9da8e3e-f099-415d-a001-23656fd425ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:03.584750 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9da8e3e-f099-415d-a001-23656fd425ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:03.585099 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9da8e3e-f099-415d-a001-23656fd425ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:03.585329 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9da8e3e-f099-415d-a001-23656fd425ba admin admin] PATH_INFO: `/v3/projects/0b3b7ef5ca3c4d72b9f1ae325f0970ef` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:03.586479 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9da8e3e-f099-415d-a001-23656fd425ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b3b7ef5ca3c4d72b9f1ae325f0970ef)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:03.588530 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9da8e3e-f099-415d-a001-23656fd425ba admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:03.707212 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-b9da8e3e-f099-415d-a001-23656fd425ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b3b7ef5ca3c4d72b9f1ae325f0970ef', 'request_id': 'req-b9da8e3e-f099-415d-a001-23656fd425ba'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:03.714256 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-b9da8e3e-f099-415d-a001-23656fd425ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b3b7ef5ca3c4d72b9f1ae325f0970ef', 'request_id': 'req-b9da8e3e-f099-415d-a001-23656fd425ba'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:03.791611 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-b9da8e3e-f099-415d-a001-23656fd425ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b3b7ef5ca3c4d72b9f1ae325f0970ef', 'request_id': 'req-b9da8e3e-f099-415d-a001-23656fd425ba'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:03.803873 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-b9da8e3e-f099-415d-a001-23656fd425ba admin admin] The token cache is being invalidate because project 0b3b7ef5ca3c4d72b9f1ae325f0970ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81747) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:28:03.804481 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-b9da8e3e-f099-415d-a001-23656fd425ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:03.805118 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-b9da8e3e-f099-415d-a001-23656fd425ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:03.806655 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1166/2321] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:28:03 2022] DELETE /identity/v3/projects/0b3b7ef5ca3c4d72b9f1ae325f0970ef => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:28:03.810866 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef147074-80ca-4ec7-88e7-40ca4a2b7752 service neutron] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:03.821888 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef147074-80ca-4ec7-88e7-40ca4a2b7752 service neutron] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:03.877790 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7875e4e-d4df-42cb-8886-f89380cdb7de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:03.878347 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7875e4e-d4df-42cb-8886-f89380cdb7de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:03.878628 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7875e4e-d4df-42cb-8886-f89380cdb7de admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:03.878899 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7875e4e-d4df-42cb-8886-f89380cdb7de admin admin] PATH_INFO: `/v3/users/dd13da8b7c3b437e875495369a6f7287` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:03.880042 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7875e4e-d4df-42cb-8886-f89380cdb7de admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd13da8b7c3b437e875495369a6f7287)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:03.882639 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7875e4e-d4df-42cb-8886-f89380cdb7de admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:03.906398 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:28:04.073708 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-b7875e4e-d4df-42cb-8886-f89380cdb7de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd13da8b7c3b437e875495369a6f7287'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:04.074363 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-b7875e4e-d4df-42cb-8886-f89380cdb7de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd13da8b7c3b437e875495369a6f7287'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:04.086469 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-b7875e4e-d4df-42cb-8886-f89380cdb7de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd13da8b7c3b437e875495369a6f7287'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:04.168848 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-b7875e4e-d4df-42cb-8886-f89380cdb7de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd13da8b7c3b437e875495369a6f7287'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:04.169665 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-b7875e4e-d4df-42cb-8886-f89380cdb7de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd13da8b7c3b437e875495369a6f7287'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:04.183840 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1156/2322] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:28:03 2022] DELETE /identity/v3/users/dd13da8b7c3b437e875495369a6f7287 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:28:04.647558 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9da8e3e-f099-415d-a001-23656fd425ba admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:04.649033 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-b9da8e3e-f099-415d-a001-23656fd425ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:28:04.662380 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9da8e3e-f099-415d-a001-23656fd425ba admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:04.721734 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489368a0-a9e8-485d-b0e4-d938b7560fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:04.722374 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489368a0-a9e8-485d-b0e4-d938b7560fe5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:04.722648 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489368a0-a9e8-485d-b0e4-d938b7560fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:04.722916 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489368a0-a9e8-485d-b0e4-d938b7560fe5 admin admin] PATH_INFO: `/v3/projects/34c86a4c72c74c9aa1981aa955764416` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:04.724183 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489368a0-a9e8-485d-b0e4-d938b7560fe5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34c86a4c72c74c9aa1981aa955764416)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:04.726912 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489368a0-a9e8-485d-b0e4-d938b7560fe5 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:04.844500 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:28:04.850601 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-489368a0-a9e8-485d-b0e4-d938b7560fe5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34c86a4c72c74c9aa1981aa955764416', 'request_id': 'req-489368a0-a9e8-485d-b0e4-d938b7560fe5'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:04.859054 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-489368a0-a9e8-485d-b0e4-d938b7560fe5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34c86a4c72c74c9aa1981aa955764416', 'request_id': 'req-489368a0-a9e8-485d-b0e4-d938b7560fe5'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:04.957310 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-489368a0-a9e8-485d-b0e4-d938b7560fe5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34c86a4c72c74c9aa1981aa955764416', 'request_id': 'req-489368a0-a9e8-485d-b0e4-d938b7560fe5'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:04.968987 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-489368a0-a9e8-485d-b0e4-d938b7560fe5 admin admin] The token cache is being invalidate because project 34c86a4c72c74c9aa1981aa955764416 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81747) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:28:04.970290 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-489368a0-a9e8-485d-b0e4-d938b7560fe5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:04.970290 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-489368a0-a9e8-485d-b0e4-d938b7560fe5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:04.971410 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1167/2323] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:28:04 2022] DELETE /identity/v3/projects/34c86a4c72c74c9aa1981aa955764416 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:28:04.976393 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08982599-b964-475f-945a-65980ec308c6 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:04.976858 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08982599-b964-475f-945a-65980ec308c6 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:04.976979 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08982599-b964-475f-945a-65980ec308c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:05.016728 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-08982599-b964-475f-945a-65980ec308c6 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:28:05.018171 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-08982599-b964-475f-945a-65980ec308c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:28:05.078653 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1157/2324] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:28:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:28:05.087095 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338c16fa-0b7c-4d4d-8a81-cb15e2ca5502 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:05.093499 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338c16fa-0b7c-4d4d-8a81-cb15e2ca5502 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:05.093499 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338c16fa-0b7c-4d4d-8a81-cb15e2ca5502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:05.106908 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-338c16fa-0b7c-4d4d-8a81-cb15e2ca5502 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:28:05.109104 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-338c16fa-0b7c-4d4d-8a81-cb15e2ca5502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:28:05.118716 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1168/2325] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:28:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:28:05.123012 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08982599-b964-475f-945a-65980ec308c6 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:05.132790 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08982599-b964-475f-945a-65980ec308c6 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:05.137380 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1ac2e9-09c1-4e9a-990b-984a61cbb48b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:05.137975 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1ac2e9-09c1-4e9a-990b-984a61cbb48b admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:05.138251 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1ac2e9-09c1-4e9a-990b-984a61cbb48b admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:05.138521 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1ac2e9-09c1-4e9a-990b-984a61cbb48b admin admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:05.139166 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1ac2e9-09c1-4e9a-990b-984a61cbb48b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:28:05.139572 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1ac2e9-09c1-4e9a-990b-984a61cbb48b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:05.141175 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1ac2e9-09c1-4e9a-990b-984a61cbb48b admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:05.153988 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1158/2326] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Oct 12 00:28:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:28:05.157598 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338c16fa-0b7c-4d4d-8a81-cb15e2ca5502 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:05.166719 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338c16fa-0b7c-4d4d-8a81-cb15e2ca5502 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:05.171597 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff55e42d-c904-4417-9aaa-580ad3019d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:05.172196 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff55e42d-c904-4417-9aaa-580ad3019d81 admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:05.172471 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff55e42d-c904-4417-9aaa-580ad3019d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:05.172738 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff55e42d-c904-4417-9aaa-580ad3019d81 admin admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:05.173426 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff55e42d-c904-4417-9aaa-580ad3019d81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:05.175308 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff55e42d-c904-4417-9aaa-580ad3019d81 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:05.244099 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1169/2327] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:28:05 2022] POST /identity/v3/projects => generated 387 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:28:05.248351 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1ac2e9-09c1-4e9a-990b-984a61cbb48b admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:05.259288 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1ac2e9-09c1-4e9a-990b-984a61cbb48b admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:05.314427 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc3a873-d4d3-43ff-9e3f-b178c204a607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:05.315024 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc3a873-d4d3-43ff-9e3f-b178c204a607 admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:05.315303 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc3a873-d4d3-43ff-9e3f-b178c204a607 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:05.315603 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc3a873-d4d3-43ff-9e3f-b178c204a607 admin admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:05.316238 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc3a873-d4d3-43ff-9e3f-b178c204a607 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:05.318767 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc3a873-d4d3-43ff-9e3f-b178c204a607 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:05.363845 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:28:05.369271 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1159/2328] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:28:05 2022] POST /identity/v3/users => generated 294 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:28:05.373489 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff55e42d-c904-4417-9aaa-580ad3019d81 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:05.384283 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff55e42d-c904-4417-9aaa-580ad3019d81 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:05.389019 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcd369f-6a85-4d43-a338-71d8e79c1ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:05.389611 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcd369f-6a85-4d43-a338-71d8e79c1ab5 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:05.389887 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcd369f-6a85-4d43-a338-71d8e79c1ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:05.390155 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcd369f-6a85-4d43-a338-71d8e79c1ab5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:05.390838 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcd369f-6a85-4d43-a338-71d8e79c1ab5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:05.392460 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcd369f-6a85-4d43-a338-71d8e79c1ab5 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:05.402457 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1170/2329] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:28:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:28:05.406255 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc3a873-d4d3-43ff-9e3f-b178c204a607 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:05.416289 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc3a873-d4d3-43ff-9e3f-b178c204a607 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:05.420940 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc329fa3-2df7-4ef6-8f3b-992574b61327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:05.421570 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc329fa3-2df7-4ef6-8f3b-992574b61327 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:05.421807 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc329fa3-2df7-4ef6-8f3b-992574b61327 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:05.422073 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc329fa3-2df7-4ef6-8f3b-992574b61327 admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:05.422703 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc329fa3-2df7-4ef6-8f3b-992574b61327 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:05.424361 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc329fa3-2df7-4ef6-8f3b-992574b61327 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:05.436018 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1160/2330] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:28:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:28:05.439924 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcd369f-6a85-4d43-a338-71d8e79c1ab5 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:05.450633 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcd369f-6a85-4d43-a338-71d8e79c1ab5 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:05.455239 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f42ef0b-bf63-45e8-82a7-add18e62038d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:05.459719 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f42ef0b-bf63-45e8-82a7-add18e62038d admin admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:05.459719 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f42ef0b-bf63-45e8-82a7-add18e62038d admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:05.459719 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f42ef0b-bf63-45e8-82a7-add18e62038d admin admin] PATH_INFO: `/v3/projects/048bc5498c85412ba5b17a88df802c05/users/4174acb5760f48cb9f5d0eb3b19ff214/roles/ef689f96442945e8a7bc8cbfee13d909` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:05.470731 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f42ef0b-bf63-45e8-82a7-add18e62038d admin admin] RBAC: Authorizing `identity:create_grant(project_id=048bc5498c85412ba5b17a88df802c05, user_id=4174acb5760f48cb9f5d0eb3b19ff214, role_id=ef689f96442945e8a7bc8cbfee13d909)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:05.473460 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f42ef0b-bf63-45e8-82a7-add18e62038d admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:05.522502 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1171/2331] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:28:05 2022] PUT /identity/v3/projects/048bc5498c85412ba5b17a88df802c05/users/4174acb5760f48cb9f5d0eb3b19ff214/roles/ef689f96442945e8a7bc8cbfee13d909 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 00:28:05.532649 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b7ffe3-f24d-4abe-85c3-2af3fe6a7055 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:05.532946 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b7ffe3-f24d-4abe-85c3-2af3fe6a7055 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:05.533243 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b7ffe3-f24d-4abe-85c3-2af3fe6a7055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:05.562369 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-a1b7ffe3-f24d-4abe-85c3-2af3fe6a7055 None None] MFA Rules not processed for user `4174acb5760f48cb9f5d0eb3b19ff214`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:28:05.563620 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-a1b7ffe3-f24d-4abe-85c3-2af3fe6a7055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:28:05.627905 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1161/2332] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:28:05 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:28:05.633108 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f42ef0b-bf63-45e8-82a7-add18e62038d admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:05.693612 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f42ef0b-bf63-45e8-82a7-add18e62038d admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:05.698316 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5755b2-b201-4b55-8866-45ebd57b0b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:05.698907 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5755b2-b201-4b55-8866-45ebd57b0b52 admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:05.699179 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5755b2-b201-4b55-8866-45ebd57b0b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:05.699473 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5755b2-b201-4b55-8866-45ebd57b0b52 admin admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:05.700088 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5755b2-b201-4b55-8866-45ebd57b0b52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:05.701959 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5755b2-b201-4b55-8866-45ebd57b0b52 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:05.796003 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1172/2333] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:28:05 2022] POST /identity/v3/projects => generated 389 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:28:05.798704 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b7ffe3-f24d-4abe-85c3-2af3fe6a7055 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:05.810337 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b7ffe3-f24d-4abe-85c3-2af3fe6a7055 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:05.873939 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e39376-0676-4d5a-a49b-bdf67926568e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:05.874134 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e39376-0676-4d5a-a49b-bdf67926568e admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:05.874415 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e39376-0676-4d5a-a49b-bdf67926568e admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:05.874717 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e39376-0676-4d5a-a49b-bdf67926568e admin admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:05.875452 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e39376-0676-4d5a-a49b-bdf67926568e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:05.877453 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e39376-0676-4d5a-a49b-bdf67926568e admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:05.958193 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1162/2334] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:28:05 2022] POST /identity/v3/users => generated 301 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 00:28:05.962082 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5755b2-b201-4b55-8866-45ebd57b0b52 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:05.970565 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5755b2-b201-4b55-8866-45ebd57b0b52 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:05.975994 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b598cff3-d7b0-4cc0-ab84-a8c7e21febc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:05.976592 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b598cff3-d7b0-4cc0-ab84-a8c7e21febc1 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:05.976869 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b598cff3-d7b0-4cc0-ab84-a8c7e21febc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:05.977140 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b598cff3-d7b0-4cc0-ab84-a8c7e21febc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:05.977778 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b598cff3-d7b0-4cc0-ab84-a8c7e21febc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:05.979483 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b598cff3-d7b0-4cc0-ab84-a8c7e21febc1 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:05.990086 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1173/2335] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:28:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:28:05.994016 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e39376-0676-4d5a-a49b-bdf67926568e admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:06.004386 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e39376-0676-4d5a-a49b-bdf67926568e admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:06.009106 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eeb4cc-7a80-45e9-9e43-a873292f1fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:06.009754 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eeb4cc-7a80-45e9-9e43-a873292f1fe8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:06.010034 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eeb4cc-7a80-45e9-9e43-a873292f1fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:06.010305 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eeb4cc-7a80-45e9-9e43-a873292f1fe8 admin admin] PATH_INFO: `/v3/projects/968ab88c91194787ba553f216cd6597e/users/ad74d6b1a94042bfacc7d5825aa8e3aa/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:06.025498 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eeb4cc-7a80-45e9-9e43-a873292f1fe8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=968ab88c91194787ba553f216cd6597e, user_id=ad74d6b1a94042bfacc7d5825aa8e3aa, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:06.028230 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eeb4cc-7a80-45e9-9e43-a873292f1fe8 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:06.134848 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1163/2336] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:28:05 2022] PUT /identity/v3/projects/968ab88c91194787ba553f216cd6597e/users/ad74d6b1a94042bfacc7d5825aa8e3aa/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 00:28:06.139008 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b598cff3-d7b0-4cc0-ab84-a8c7e21febc1 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:06.149685 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b598cff3-d7b0-4cc0-ab84-a8c7e21febc1 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:06.190412 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:28:06.210467 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ff0a9b-11e8-45ab-8912-b0537cfb3d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:06.211084 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ff0a9b-11e8-45ab-8912-b0537cfb3d4d admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:06.211386 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ff0a9b-11e8-45ab-8912-b0537cfb3d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:06.211673 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ff0a9b-11e8-45ab-8912-b0537cfb3d4d admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:06.212342 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ff0a9b-11e8-45ab-8912-b0537cfb3d4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:06.214057 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ff0a9b-11e8-45ab-8912-b0537cfb3d4d admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:06.226559 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1174/2337] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:28:06 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:28:06.231451 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eeb4cc-7a80-45e9-9e43-a873292f1fe8 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:06.242083 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eeb4cc-7a80-45e9-9e43-a873292f1fe8 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:06.247000 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8fe7e8-aa34-4512-b4f7-cbb934b05b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:06.247755 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8fe7e8-aa34-4512-b4f7-cbb934b05b47 admin admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:06.248017 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8fe7e8-aa34-4512-b4f7-cbb934b05b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:06.248282 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8fe7e8-aa34-4512-b4f7-cbb934b05b47 admin admin] PATH_INFO: `/v3/projects/968ab88c91194787ba553f216cd6597e/users/ad74d6b1a94042bfacc7d5825aa8e3aa/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:06.250311 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8fe7e8-aa34-4512-b4f7-cbb934b05b47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=968ab88c91194787ba553f216cd6597e, user_id=ad74d6b1a94042bfacc7d5825aa8e3aa, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:06.253037 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8fe7e8-aa34-4512-b4f7-cbb934b05b47 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:06.264110 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1164/2338] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:28:06 2022] PUT /identity/v3/projects/968ab88c91194787ba553f216cd6597e/users/ad74d6b1a94042bfacc7d5825aa8e3aa/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:28:06.281022 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b57e67-ca4d-41c8-968c-62f0a0b5e205 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:06.281302 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b57e67-ca4d-41c8-968c-62f0a0b5e205 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:06.281570 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b57e67-ca4d-41c8-968c-62f0a0b5e205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:06.309412 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-a3b57e67-ca4d-41c8-968c-62f0a0b5e205 None None] MFA Rules not processed for user `ad74d6b1a94042bfacc7d5825aa8e3aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:28:06.310581 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-a3b57e67-ca4d-41c8-968c-62f0a0b5e205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:28:06.378295 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1175/2339] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:28:06 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:28:06.389763 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8fe7e8-aa34-4512-b4f7-cbb934b05b47 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:06.391466 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-fa8fe7e8-aa34-4512-b4f7-cbb934b05b47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:28:06.426800 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:28:06.450499 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8fe7e8-aa34-4512-b4f7-cbb934b05b47 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:06.454193 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b471bf-cc77-4361-80de-32d3c727f513 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:06.454812 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b471bf-cc77-4361-80de-32d3c727f513 service cinder] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:06.455085 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b471bf-cc77-4361-80de-32d3c727f513 service cinder] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:06.455352 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b471bf-cc77-4361-80de-32d3c727f513 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:06.464836 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b471bf-cc77-4361-80de-32d3c727f513 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:06.467336 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b471bf-cc77-4361-80de-32d3c727f513 service cinder] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:06.476296 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1165/2340] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:28:06 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:28:06.610220 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b57e67-ca4d-41c8-968c-62f0a0b5e205 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:06.670274 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b57e67-ca4d-41c8-968c-62f0a0b5e205 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:06.674700 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a580eb7-38da-4f64-a069-410a70c83cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:06.675316 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a580eb7-38da-4f64-a069-410a70c83cc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:06.675616 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a580eb7-38da-4f64-a069-410a70c83cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:06.675883 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a580eb7-38da-4f64-a069-410a70c83cc2 admin admin] PATH_INFO: `/v3/users/4174acb5760f48cb9f5d0eb3b19ff214` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:06.676973 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a580eb7-38da-4f64-a069-410a70c83cc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4174acb5760f48cb9f5d0eb3b19ff214)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:06.679436 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a580eb7-38da-4f64-a069-410a70c83cc2 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:06.861936 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-1a580eb7-38da-4f64-a069-410a70c83cc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4174acb5760f48cb9f5d0eb3b19ff214'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:06.862612 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-1a580eb7-38da-4f64-a069-410a70c83cc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4174acb5760f48cb9f5d0eb3b19ff214'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:06.873983 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:28:06.875247 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-1a580eb7-38da-4f64-a069-410a70c83cc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4174acb5760f48cb9f5d0eb3b19ff214'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:06.945949 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-1a580eb7-38da-4f64-a069-410a70c83cc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4174acb5760f48cb9f5d0eb3b19ff214'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:06.946692 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-1a580eb7-38da-4f64-a069-410a70c83cc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4174acb5760f48cb9f5d0eb3b19ff214'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:06.962903 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1176/2341] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:28:06 2022] DELETE /identity/v3/users/4174acb5760f48cb9f5d0eb3b19ff214 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:28:06.964519 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574895bb-d6cc-4b15-b650-2296cc21cd2d None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:06.969952 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574895bb-d6cc-4b15-b650-2296cc21cd2d None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:06.969952 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574895bb-d6cc-4b15-b650-2296cc21cd2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:06.984088 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-574895bb-d6cc-4b15-b650-2296cc21cd2d None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:28:06.985296 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-574895bb-d6cc-4b15-b650-2296cc21cd2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:28:07.048531 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1166/2342] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:28:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:28:07.058960 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a580eb7-38da-4f64-a069-410a70c83cc2 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:07.060437 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-1a580eb7-38da-4f64-a069-410a70c83cc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:28:07.118606 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a580eb7-38da-4f64-a069-410a70c83cc2 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:07.122831 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bf4842-be8f-4870-a700-54a59eed8f49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:07.123486 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf4842-be8f-4870-a700-54a59eed8f49 service neutron] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:07.123763 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf4842-be8f-4870-a700-54a59eed8f49 service neutron] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:07.124031 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf4842-be8f-4870-a700-54a59eed8f49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:07.133732 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bf4842-be8f-4870-a700-54a59eed8f49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:07.135147 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bf4842-be8f-4870-a700-54a59eed8f49 service neutron] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:07.144744 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1177/2343] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:28:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:28:07.489908 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574895bb-d6cc-4b15-b650-2296cc21cd2d None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:07.496572 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-574895bb-d6cc-4b15-b650-2296cc21cd2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:28:07.503643 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574895bb-d6cc-4b15-b650-2296cc21cd2d None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:07.508417 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873ee3da-4705-4771-bb7e-4dda6b9e3368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:07.509050 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ee3da-4705-4771-bb7e-4dda6b9e3368 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:07.509328 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ee3da-4705-4771-bb7e-4dda6b9e3368 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:07.509602 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ee3da-4705-4771-bb7e-4dda6b9e3368 admin admin] PATH_INFO: `/v3/projects/048bc5498c85412ba5b17a88df802c05` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:07.510769 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873ee3da-4705-4771-bb7e-4dda6b9e3368 admin admin] RBAC: Authorizing `identity:delete_project(project_id=048bc5498c85412ba5b17a88df802c05)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:07.512711 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873ee3da-4705-4771-bb7e-4dda6b9e3368 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:07.814924 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-873ee3da-4705-4771-bb7e-4dda6b9e3368 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '048bc5498c85412ba5b17a88df802c05', 'request_id': 'req-873ee3da-4705-4771-bb7e-4dda6b9e3368'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:07.821821 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-873ee3da-4705-4771-bb7e-4dda6b9e3368 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '048bc5498c85412ba5b17a88df802c05', 'request_id': 'req-873ee3da-4705-4771-bb7e-4dda6b9e3368'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:07.923007 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-873ee3da-4705-4771-bb7e-4dda6b9e3368 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '048bc5498c85412ba5b17a88df802c05', 'request_id': 'req-873ee3da-4705-4771-bb7e-4dda6b9e3368'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:07.933901 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-873ee3da-4705-4771-bb7e-4dda6b9e3368 admin admin] The token cache is being invalidate because project 048bc5498c85412ba5b17a88df802c05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81746) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:28:07.934204 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-873ee3da-4705-4771-bb7e-4dda6b9e3368 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:07.934823 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-873ee3da-4705-4771-bb7e-4dda6b9e3368 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:07.936463 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1167/2344] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:28:07 2022] DELETE /identity/v3/projects/048bc5498c85412ba5b17a88df802c05 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:28:07.940438 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bf4842-be8f-4870-a700-54a59eed8f49 service neutron] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:07.950511 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bf4842-be8f-4870-a700-54a59eed8f49 service neutron] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:08.007131 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee835bbe-d71d-4b5e-b217-6102fb26badb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:08.007821 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee835bbe-d71d-4b5e-b217-6102fb26badb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:08.008096 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee835bbe-d71d-4b5e-b217-6102fb26badb admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:08.008369 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee835bbe-d71d-4b5e-b217-6102fb26badb admin admin] PATH_INFO: `/v3/users/ad74d6b1a94042bfacc7d5825aa8e3aa` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:08.009479 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee835bbe-d71d-4b5e-b217-6102fb26badb admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad74d6b1a94042bfacc7d5825aa8e3aa)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:08.011536 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee835bbe-d71d-4b5e-b217-6102fb26badb admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:08.177986 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-ee835bbe-d71d-4b5e-b217-6102fb26badb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad74d6b1a94042bfacc7d5825aa8e3aa'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:08.178692 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-ee835bbe-d71d-4b5e-b217-6102fb26badb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad74d6b1a94042bfacc7d5825aa8e3aa'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:08.191884 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-ee835bbe-d71d-4b5e-b217-6102fb26badb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad74d6b1a94042bfacc7d5825aa8e3aa'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:08.272257 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-ee835bbe-d71d-4b5e-b217-6102fb26badb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad74d6b1a94042bfacc7d5825aa8e3aa'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:08.273349 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-ee835bbe-d71d-4b5e-b217-6102fb26badb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad74d6b1a94042bfacc7d5825aa8e3aa'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:08.286616 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1178/2345] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:28:07 2022] DELETE /identity/v3/users/ad74d6b1a94042bfacc7d5825aa8e3aa => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 00:28:08.723514 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873ee3da-4705-4771-bb7e-4dda6b9e3368 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:08.725004 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-873ee3da-4705-4771-bb7e-4dda6b9e3368 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:28:08.738281 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873ee3da-4705-4771-bb7e-4dda6b9e3368 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:08.792792 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:28:08.798727 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ae988e-17e3-49b2-b304-a6ba113e3d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:08.799376 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ae988e-17e3-49b2-b304-a6ba113e3d4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:08.799678 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ae988e-17e3-49b2-b304-a6ba113e3d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:08.800060 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ae988e-17e3-49b2-b304-a6ba113e3d4e admin admin] PATH_INFO: `/v3/projects/968ab88c91194787ba553f216cd6597e` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:08.801323 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ae988e-17e3-49b2-b304-a6ba113e3d4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=968ab88c91194787ba553f216cd6597e)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:08.803265 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ae988e-17e3-49b2-b304-a6ba113e3d4e admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:08.912566 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-31ae988e-17e3-49b2-b304-a6ba113e3d4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '968ab88c91194787ba553f216cd6597e', 'request_id': 'req-31ae988e-17e3-49b2-b304-a6ba113e3d4e'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:08.918708 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-31ae988e-17e3-49b2-b304-a6ba113e3d4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '968ab88c91194787ba553f216cd6597e', 'request_id': 'req-31ae988e-17e3-49b2-b304-a6ba113e3d4e'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:09.008776 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-31ae988e-17e3-49b2-b304-a6ba113e3d4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '968ab88c91194787ba553f216cd6597e', 'request_id': 'req-31ae988e-17e3-49b2-b304-a6ba113e3d4e'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:09.019694 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-31ae988e-17e3-49b2-b304-a6ba113e3d4e admin admin] The token cache is being invalidate because project 968ab88c91194787ba553f216cd6597e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81746) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:28:09.019980 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-31ae988e-17e3-49b2-b304-a6ba113e3d4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:09.020602 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-31ae988e-17e3-49b2-b304-a6ba113e3d4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:09.022093 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1168/2346] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:28:08 2022] DELETE /identity/v3/projects/968ab88c91194787ba553f216cd6597e => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:28:09.027632 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7d2aa8-632b-4c63-ac29-51ccdc97a26a None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:09.028114 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7d2aa8-632b-4c63-ac29-51ccdc97a26a None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:09.028215 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7d2aa8-632b-4c63-ac29-51ccdc97a26a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:09.035597 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:28:09.047460 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-2a7d2aa8-632b-4c63-ac29-51ccdc97a26a None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:28:09.048661 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-2a7d2aa8-632b-4c63-ac29-51ccdc97a26a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:28:09.108247 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1179/2347] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:28:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:28:09.116835 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61922a57-3bfe-4d2d-9c22-b2f488f82d3f None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:09.117212 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61922a57-3bfe-4d2d-9c22-b2f488f82d3f None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:09.117508 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61922a57-3bfe-4d2d-9c22-b2f488f82d3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:09.137629 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-61922a57-3bfe-4d2d-9c22-b2f488f82d3f None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:28:09.138940 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-61922a57-3bfe-4d2d-9c22-b2f488f82d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:28:09.147058 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1169/2348] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:28:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:28:09.151274 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7d2aa8-632b-4c63-ac29-51ccdc97a26a None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:09.161000 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7d2aa8-632b-4c63-ac29-51ccdc97a26a None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:09.165615 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ecb2fb-d5b3-40ad-9ce5-7ee0c4857998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:09.166211 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ecb2fb-d5b3-40ad-9ce5-7ee0c4857998 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:09.166488 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ecb2fb-d5b3-40ad-9ce5-7ee0c4857998 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:09.166755 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ecb2fb-d5b3-40ad-9ce5-7ee0c4857998 admin admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:09.167482 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ecb2fb-d5b3-40ad-9ce5-7ee0c4857998 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:28:09.167782 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ecb2fb-d5b3-40ad-9ce5-7ee0c4857998 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:09.169360 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ecb2fb-d5b3-40ad-9ce5-7ee0c4857998 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:09.181322 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1180/2349] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Oct 12 00:28:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:28:09.185703 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61922a57-3bfe-4d2d-9c22-b2f488f82d3f None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:09.195485 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61922a57-3bfe-4d2d-9c22-b2f488f82d3f None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:09.200070 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1493e3-c3d4-4bbd-9ba2-d1276ec6d97a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:09.200657 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1493e3-c3d4-4bbd-9ba2-d1276ec6d97a admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:09.200929 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1493e3-c3d4-4bbd-9ba2-d1276ec6d97a admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:09.201198 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1493e3-c3d4-4bbd-9ba2-d1276ec6d97a admin admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:09.201859 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1493e3-c3d4-4bbd-9ba2-d1276ec6d97a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:09.203857 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1493e3-c3d4-4bbd-9ba2-d1276ec6d97a admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:09.307818 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1170/2350] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:28:09 2022] POST /identity/v3/projects => generated 371 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:28:09.313162 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ecb2fb-d5b3-40ad-9ce5-7ee0c4857998 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:09.324707 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ecb2fb-d5b3-40ad-9ce5-7ee0c4857998 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:09.383165 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c0c031-fde1-4079-9bb6-4dd30f043e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:09.383810 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c0c031-fde1-4079-9bb6-4dd30f043e62 admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:09.384108 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c0c031-fde1-4079-9bb6-4dd30f043e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:09.384376 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c0c031-fde1-4079-9bb6-4dd30f043e62 admin admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:09.385045 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c0c031-fde1-4079-9bb6-4dd30f043e62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:09.386952 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c0c031-fde1-4079-9bb6-4dd30f043e62 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:09.457549 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1181/2351] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:28:09 2022] POST /identity/v3/users => generated 286 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:28:09.459489 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1493e3-c3d4-4bbd-9ba2-d1276ec6d97a admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:09.470287 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1493e3-c3d4-4bbd-9ba2-d1276ec6d97a admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:09.475638 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768016ba-a441-4d29-805c-3bb028e0cf30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:09.477633 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768016ba-a441-4d29-805c-3bb028e0cf30 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:09.477633 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768016ba-a441-4d29-805c-3bb028e0cf30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:09.477633 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768016ba-a441-4d29-805c-3bb028e0cf30 admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:09.477633 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768016ba-a441-4d29-805c-3bb028e0cf30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:09.479114 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768016ba-a441-4d29-805c-3bb028e0cf30 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:09.489303 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1171/2352] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:28:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:28:09.493897 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c0c031-fde1-4079-9bb6-4dd30f043e62 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:09.503968 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c0c031-fde1-4079-9bb6-4dd30f043e62 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:09.508545 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc49853-b68c-434f-9d36-36e86eb5a463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:09.509141 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc49853-b68c-434f-9d36-36e86eb5a463 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:09.509419 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc49853-b68c-434f-9d36-36e86eb5a463 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:09.509694 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc49853-b68c-434f-9d36-36e86eb5a463 admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:09.510331 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc49853-b68c-434f-9d36-36e86eb5a463 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:09.511946 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc49853-b68c-434f-9d36-36e86eb5a463 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:09.522715 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1182/2353] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:28:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:28:09.526734 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768016ba-a441-4d29-805c-3bb028e0cf30 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:09.537371 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768016ba-a441-4d29-805c-3bb028e0cf30 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:09.541974 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc61b09-c3cc-4517-a66c-a839ed4f3284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:09.542633 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc61b09-c3cc-4517-a66c-a839ed4f3284 admin admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:09.542910 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc61b09-c3cc-4517-a66c-a839ed4f3284 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:09.543180 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc61b09-c3cc-4517-a66c-a839ed4f3284 admin admin] PATH_INFO: `/v3/projects/5b74d1467e4c4cbc89e40f3539520682/users/6f3f4e07193b4f82b8d3022e3365e0ea/roles/ef689f96442945e8a7bc8cbfee13d909` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:09.558898 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc61b09-c3cc-4517-a66c-a839ed4f3284 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b74d1467e4c4cbc89e40f3539520682, user_id=6f3f4e07193b4f82b8d3022e3365e0ea, role_id=ef689f96442945e8a7bc8cbfee13d909)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:09.561634 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc61b09-c3cc-4517-a66c-a839ed4f3284 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:09.631477 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1172/2354] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:28:09 2022] PUT /identity/v3/projects/5b74d1467e4c4cbc89e40f3539520682/users/6f3f4e07193b4f82b8d3022e3365e0ea/roles/ef689f96442945e8a7bc8cbfee13d909 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:28:09.641696 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43268df1-9ea0-45af-976c-73874d29c4ca None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:09.642001 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43268df1-9ea0-45af-976c-73874d29c4ca None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:09.642272 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43268df1-9ea0-45af-976c-73874d29c4ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:09.671123 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-43268df1-9ea0-45af-976c-73874d29c4ca None None] MFA Rules not processed for user `6f3f4e07193b4f82b8d3022e3365e0ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:28:09.676720 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-43268df1-9ea0-45af-976c-73874d29c4ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:28:09.734777 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:28:09.740236 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1183/2355] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:28:09 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:28:09.751314 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc61b09-c3cc-4517-a66c-a839ed4f3284 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:09.753086 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-8cc61b09-c3cc-4517-a66c-a839ed4f3284 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:28:09.812717 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc61b09-c3cc-4517-a66c-a839ed4f3284 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:09.816491 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ba4e43-1b56-4e2e-9598-79a84957987c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:09.817100 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ba4e43-1b56-4e2e-9598-79a84957987c service cinder] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:09.817373 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ba4e43-1b56-4e2e-9598-79a84957987c service cinder] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:09.817670 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ba4e43-1b56-4e2e-9598-79a84957987c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:09.826142 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ba4e43-1b56-4e2e-9598-79a84957987c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:09.827602 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ba4e43-1b56-4e2e-9598-79a84957987c service cinder] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:09.837286 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1173/2356] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:28:09 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:28:09.863963 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43268df1-9ea0-45af-976c-73874d29c4ca None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:09.928444 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43268df1-9ea0-45af-976c-73874d29c4ca None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:09.933094 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29ccbf8-76bc-4fd8-817f-c4ee43f4def7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:09.933721 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29ccbf8-76bc-4fd8-817f-c4ee43f4def7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:09.933996 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29ccbf8-76bc-4fd8-817f-c4ee43f4def7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:09.934266 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29ccbf8-76bc-4fd8-817f-c4ee43f4def7 admin admin] PATH_INFO: `/v3/users/6f3f4e07193b4f82b8d3022e3365e0ea` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:09.935421 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29ccbf8-76bc-4fd8-817f-c4ee43f4def7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f3f4e07193b4f82b8d3022e3365e0ea)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:09.937331 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29ccbf8-76bc-4fd8-817f-c4ee43f4def7 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:10.124061 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-a29ccbf8-76bc-4fd8-817f-c4ee43f4def7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f3f4e07193b4f82b8d3022e3365e0ea'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:10.124710 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-a29ccbf8-76bc-4fd8-817f-c4ee43f4def7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f3f4e07193b4f82b8d3022e3365e0ea'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:10.137504 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-a29ccbf8-76bc-4fd8-817f-c4ee43f4def7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f3f4e07193b4f82b8d3022e3365e0ea'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:10.214426 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-a29ccbf8-76bc-4fd8-817f-c4ee43f4def7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f3f4e07193b4f82b8d3022e3365e0ea'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:10.215074 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-a29ccbf8-76bc-4fd8-817f-c4ee43f4def7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f3f4e07193b4f82b8d3022e3365e0ea'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:10.227981 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1184/2357] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:28:09 2022] DELETE /identity/v3/users/6f3f4e07193b4f82b8d3022e3365e0ea => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:28:10.232559 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed71c8c-c9f7-4efc-a5a3-ca9039e097a6 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:10.232846 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed71c8c-c9f7-4efc-a5a3-ca9039e097a6 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:10.233133 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed71c8c-c9f7-4efc-a5a3-ca9039e097a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:10.252003 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-9ed71c8c-c9f7-4efc-a5a3-ca9039e097a6 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:28:10.253191 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-9ed71c8c-c9f7-4efc-a5a3-ca9039e097a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:28:10.273967 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:28:10.311919 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1174/2358] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:28:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:28:10.322677 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29ccbf8-76bc-4fd8-817f-c4ee43f4def7 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:10.324188 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-a29ccbf8-76bc-4fd8-817f-c4ee43f4def7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:28:10.380924 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29ccbf8-76bc-4fd8-817f-c4ee43f4def7 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:10.384455 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbacf31f-f1fc-4892-8995-ebcf451c7847 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:10.385177 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbacf31f-f1fc-4892-8995-ebcf451c7847 service neutron] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:10.385449 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbacf31f-f1fc-4892-8995-ebcf451c7847 service neutron] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:10.385720 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbacf31f-f1fc-4892-8995-ebcf451c7847 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:10.397010 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbacf31f-f1fc-4892-8995-ebcf451c7847 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:10.398433 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbacf31f-f1fc-4892-8995-ebcf451c7847 service neutron] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:10.408302 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1185/2359] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:28:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:28:10.793841 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed71c8c-c9f7-4efc-a5a3-ca9039e097a6 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:10.795288 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-9ed71c8c-c9f7-4efc-a5a3-ca9039e097a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:28:10.808196 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed71c8c-c9f7-4efc-a5a3-ca9039e097a6 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:10.812998 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c9296a-cac4-4ae3-a3a3-b5852aa0f1ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:10.813578 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c9296a-cac4-4ae3-a3a3-b5852aa0f1ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:10.814284 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c9296a-cac4-4ae3-a3a3-b5852aa0f1ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:10.814499 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c9296a-cac4-4ae3-a3a3-b5852aa0f1ca admin admin] PATH_INFO: `/v3/projects/5b74d1467e4c4cbc89e40f3539520682` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:10.815702 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c9296a-cac4-4ae3-a3a3-b5852aa0f1ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b74d1467e4c4cbc89e40f3539520682)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:10.817645 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c9296a-cac4-4ae3-a3a3-b5852aa0f1ca admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:10.975579 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-c0c9296a-cac4-4ae3-a3a3-b5852aa0f1ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b74d1467e4c4cbc89e40f3539520682', 'request_id': 'req-c0c9296a-cac4-4ae3-a3a3-b5852aa0f1ca'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:10.981766 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-c0c9296a-cac4-4ae3-a3a3-b5852aa0f1ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b74d1467e4c4cbc89e40f3539520682', 'request_id': 'req-c0c9296a-cac4-4ae3-a3a3-b5852aa0f1ca'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:11.060210 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-c0c9296a-cac4-4ae3-a3a3-b5852aa0f1ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b74d1467e4c4cbc89e40f3539520682', 'request_id': 'req-c0c9296a-cac4-4ae3-a3a3-b5852aa0f1ca'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:11.070464 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-c0c9296a-cac4-4ae3-a3a3-b5852aa0f1ca admin admin] The token cache is being invalidate because project 5b74d1467e4c4cbc89e40f3539520682 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81746) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:28:11.070749 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-c0c9296a-cac4-4ae3-a3a3-b5852aa0f1ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:11.075986 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-c0c9296a-cac4-4ae3-a3a3-b5852aa0f1ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:11.075986 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1175/2360] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:28:10 2022] DELETE /identity/v3/projects/5b74d1467e4c4cbc89e40f3539520682 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:28:11.077778 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72147c2a-d8bd-4a41-97d6-459484927be3 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:11.078086 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72147c2a-d8bd-4a41-97d6-459484927be3 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:11.078364 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72147c2a-d8bd-4a41-97d6-459484927be3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:11.098126 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-72147c2a-d8bd-4a41-97d6-459484927be3 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:28:11.099851 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-72147c2a-d8bd-4a41-97d6-459484927be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:28:11.112561 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:28:11.161029 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1186/2361] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:28:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 00:28:11.227013 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11464f75-d784-4027-a8c2-5c272a347339 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:11.227224 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11464f75-d784-4027-a8c2-5c272a347339 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:11.227509 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11464f75-d784-4027-a8c2-5c272a347339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:11.247159 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-11464f75-d784-4027-a8c2-5c272a347339 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:28:11.248387 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-11464f75-d784-4027-a8c2-5c272a347339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:28:11.256512 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1176/2362] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:28:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:28:11.260660 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72147c2a-d8bd-4a41-97d6-459484927be3 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:11.270453 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72147c2a-d8bd-4a41-97d6-459484927be3 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:11.275691 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e5f5c1-bf07-4f6a-bcb0-2c07cce9088d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:11.276285 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e5f5c1-bf07-4f6a-bcb0-2c07cce9088d admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:11.276557 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e5f5c1-bf07-4f6a-bcb0-2c07cce9088d admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:11.276824 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e5f5c1-bf07-4f6a-bcb0-2c07cce9088d admin admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:11.277459 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e5f5c1-bf07-4f6a-bcb0-2c07cce9088d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:28:11.277755 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e5f5c1-bf07-4f6a-bcb0-2c07cce9088d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:11.279348 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e5f5c1-bf07-4f6a-bcb0-2c07cce9088d admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:11.291416 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1187/2363] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Oct 12 00:28:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:28:11.295615 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11464f75-d784-4027-a8c2-5c272a347339 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:11.305630 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11464f75-d784-4027-a8c2-5c272a347339 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:11.310315 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbd7167-c267-42f3-8e60-6f18544b8b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:11.312991 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd7167-c267-42f3-8e60-6f18544b8b0a admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:11.312991 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd7167-c267-42f3-8e60-6f18544b8b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:11.312991 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd7167-c267-42f3-8e60-6f18544b8b0a admin admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:11.312991 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbd7167-c267-42f3-8e60-6f18544b8b0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:11.314000 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbd7167-c267-42f3-8e60-6f18544b8b0a admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:11.386775 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1177/2364] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:28:11 2022] POST /identity/v3/projects => generated 375 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:28:11.391020 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e5f5c1-bf07-4f6a-bcb0-2c07cce9088d admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:11.401754 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e5f5c1-bf07-4f6a-bcb0-2c07cce9088d admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:11.462852 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8df7770-8daa-4cd7-af4a-7fdaf1f3005b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:11.463496 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8df7770-8daa-4cd7-af4a-7fdaf1f3005b admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:11.463778 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8df7770-8daa-4cd7-af4a-7fdaf1f3005b admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:11.464047 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8df7770-8daa-4cd7-af4a-7fdaf1f3005b admin admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:11.464690 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8df7770-8daa-4cd7-af4a-7fdaf1f3005b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:11.466598 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8df7770-8daa-4cd7-af4a-7fdaf1f3005b admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:11.562246 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1188/2365] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:28:11 2022] POST /identity/v3/users => generated 288 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:28:11.566293 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbd7167-c267-42f3-8e60-6f18544b8b0a admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:11.577003 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbd7167-c267-42f3-8e60-6f18544b8b0a admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:11.581999 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f008144e-52f3-487e-8816-824a95c40a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:11.582590 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f008144e-52f3-487e-8816-824a95c40a61 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:11.582866 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f008144e-52f3-487e-8816-824a95c40a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:11.583141 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f008144e-52f3-487e-8816-824a95c40a61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:11.584497 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f008144e-52f3-487e-8816-824a95c40a61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:11.586087 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f008144e-52f3-487e-8816-824a95c40a61 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:11.596630 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1178/2366] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:28:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:28:11.600480 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8df7770-8daa-4cd7-af4a-7fdaf1f3005b admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:11.610353 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8df7770-8daa-4cd7-af4a-7fdaf1f3005b admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:11.614946 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78808e3d-95e9-4c8c-9b5c-dcdb2408687c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:11.615569 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78808e3d-95e9-4c8c-9b5c-dcdb2408687c admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:11.615847 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78808e3d-95e9-4c8c-9b5c-dcdb2408687c admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:11.616117 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78808e3d-95e9-4c8c-9b5c-dcdb2408687c admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:11.616748 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78808e3d-95e9-4c8c-9b5c-dcdb2408687c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:11.618341 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78808e3d-95e9-4c8c-9b5c-dcdb2408687c admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:11.630992 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1189/2367] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:28:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 00:28:11.633130 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f008144e-52f3-487e-8816-824a95c40a61 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:11.643148 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f008144e-52f3-487e-8816-824a95c40a61 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:11.648541 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c9c70a-6b4a-4daf-aafc-9bafef907d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:11.649189 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c9c70a-6b4a-4daf-aafc-9bafef907d34 admin admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:11.649466 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c9c70a-6b4a-4daf-aafc-9bafef907d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:11.649737 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c9c70a-6b4a-4daf-aafc-9bafef907d34 admin admin] PATH_INFO: `/v3/projects/90c1e5e7db4942dea48cd03d186bb033/users/0579e56a0b0b434389706f8f0b236ce3/roles/ef689f96442945e8a7bc8cbfee13d909` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:11.664344 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c9c70a-6b4a-4daf-aafc-9bafef907d34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90c1e5e7db4942dea48cd03d186bb033, user_id=0579e56a0b0b434389706f8f0b236ce3, role_id=ef689f96442945e8a7bc8cbfee13d909)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:11.666992 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c9c70a-6b4a-4daf-aafc-9bafef907d34 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:11.792075 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1179/2368] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:28:11 2022] PUT /identity/v3/projects/90c1e5e7db4942dea48cd03d186bb033/users/0579e56a0b0b434389706f8f0b236ce3/roles/ef689f96442945e8a7bc8cbfee13d909 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:28:11.801985 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3548aa3-1ba4-4c0e-ae0c-ad8ab7192c52 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:11.802203 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3548aa3-1ba4-4c0e-ae0c-ad8ab7192c52 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:11.802472 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3548aa3-1ba4-4c0e-ae0c-ad8ab7192c52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:11.830697 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-e3548aa3-1ba4-4c0e-ae0c-ad8ab7192c52 None None] MFA Rules not processed for user `0579e56a0b0b434389706f8f0b236ce3`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:28:11.831926 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-e3548aa3-1ba4-4c0e-ae0c-ad8ab7192c52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:28:11.896555 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1190/2369] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:28:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:28:11.906908 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c9c70a-6b4a-4daf-aafc-9bafef907d34 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:11.908615 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-68c9c70a-6b4a-4daf-aafc-9bafef907d34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:28:11.960389 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:28:11.965656 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c9c70a-6b4a-4daf-aafc-9bafef907d34 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:11.969312 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fca8e3-6d0f-43a3-98b1-ef871a04cda0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:11.981338 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fca8e3-6d0f-43a3-98b1-ef871a04cda0 service cinder] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:11.981338 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fca8e3-6d0f-43a3-98b1-ef871a04cda0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:11.981338 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fca8e3-6d0f-43a3-98b1-ef871a04cda0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:11.981338 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fca8e3-6d0f-43a3-98b1-ef871a04cda0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:11.981338 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fca8e3-6d0f-43a3-98b1-ef871a04cda0 service cinder] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:11.988351 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1180/2370] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:28:11 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:28:14.644589 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560fe60f-c5ae-41bf-91e3-5b62ea746c77 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:14.645170 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560fe60f-c5ae-41bf-91e3-5b62ea746c77 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:14.645170 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560fe60f-c5ae-41bf-91e3-5b62ea746c77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:14.664239 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-560fe60f-c5ae-41bf-91e3-5b62ea746c77 None None] MFA Rules not processed for user `0579e56a0b0b434389706f8f0b236ce3`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:28:14.665448 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-560fe60f-c5ae-41bf-91e3-5b62ea746c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:28:14.672545 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1191/2371] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:28:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:28:16.790219 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fca8e3-6d0f-43a3-98b1-ef871a04cda0 service cinder] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:16.791751 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-15fca8e3-6d0f-43a3-98b1-ef871a04cda0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:28:16.886936 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fca8e3-6d0f-43a3-98b1-ef871a04cda0 service cinder] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:16.892034 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743d513d-e0a1-4d6c-a704-53c72b0e3c9b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e528da752ed4c78bc747215909f6569', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:16.892632 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d513d-e0a1-4d6c-a704-53c72b0e3c9b service placement] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:16.893181 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d513d-e0a1-4d6c-a704-53c72b0e3c9b service placement] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:16.893459 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d513d-e0a1-4d6c-a704-53c72b0e3c9b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:16.895199 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-743d513d-e0a1-4d6c-a704-53c72b0e3c9b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:28:16.906502 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743d513d-e0a1-4d6c-a704-53c72b0e3c9b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:16.908174 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743d513d-e0a1-4d6c-a704-53c72b0e3c9b service placement] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:16.918628 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1181/2372] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:28:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:28:21.576972 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560fe60f-c5ae-41bf-91e3-5b62ea746c77 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:21.578409 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-560fe60f-c5ae-41bf-91e3-5b62ea746c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:28:21.589081 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:28:21.634340 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560fe60f-c5ae-41bf-91e3-5b62ea746c77 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:21.638685 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cdfbf0-8f05-4dbf-b71b-b0e2861dbc92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:21.639292 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cdfbf0-8f05-4dbf-b71b-b0e2861dbc92 service neutron] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:21.639590 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cdfbf0-8f05-4dbf-b71b-b0e2861dbc92 service neutron] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:21.639858 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cdfbf0-8f05-4dbf-b71b-b0e2861dbc92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:21.641563 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-32cdfbf0-8f05-4dbf-b71b-b0e2861dbc92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:28:21.703988 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cdfbf0-8f05-4dbf-b71b-b0e2861dbc92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:21.705403 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cdfbf0-8f05-4dbf-b71b-b0e2861dbc92 service neutron] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:21.714798 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1192/2373] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:28:21 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:28:26.914348 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=46 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:28:31.710585 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=46 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:28:33.898105 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743d513d-e0a1-4d6c-a704-53c72b0e3c9b service placement] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:33.899593 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-743d513d-e0a1-4d6c-a704-53c72b0e3c9b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:28:33.976080 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743d513d-e0a1-4d6c-a704-53c72b0e3c9b service placement] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:33.981068 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8013b3b3-c8e0-4f58-a87a-9837bd4caeda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:33.981710 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8013b3b3-c8e0-4f58-a87a-9837bd4caeda admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:33.981988 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8013b3b3-c8e0-4f58-a87a-9837bd4caeda admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:33.982260 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8013b3b3-c8e0-4f58-a87a-9837bd4caeda admin admin] PATH_INFO: `/v3/users/6f3d89766f3a4ac0866dd2b69a029777` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:33.983499 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8013b3b3-c8e0-4f58-a87a-9837bd4caeda admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f3d89766f3a4ac0866dd2b69a029777)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:33.985538 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8013b3b3-c8e0-4f58-a87a-9837bd4caeda admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:34.152770 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-8013b3b3-c8e0-4f58-a87a-9837bd4caeda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f3d89766f3a4ac0866dd2b69a029777'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:34.153982 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-8013b3b3-c8e0-4f58-a87a-9837bd4caeda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f3d89766f3a4ac0866dd2b69a029777'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:34.181348 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-8013b3b3-c8e0-4f58-a87a-9837bd4caeda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f3d89766f3a4ac0866dd2b69a029777'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:34.255573 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-8013b3b3-c8e0-4f58-a87a-9837bd4caeda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f3d89766f3a4ac0866dd2b69a029777'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:34.256189 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-8013b3b3-c8e0-4f58-a87a-9837bd4caeda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f3d89766f3a4ac0866dd2b69a029777'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:34.270662 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1182/2374] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:28:33 2022] DELETE /identity/v3/users/6f3d89766f3a4ac0866dd2b69a029777 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:28:34.275203 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23207569-dbbe-405b-ba61-28913d3477f9 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:34.275756 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23207569-dbbe-405b-ba61-28913d3477f9 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:34.277766 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23207569-dbbe-405b-ba61-28913d3477f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:34.307434 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-23207569-dbbe-405b-ba61-28913d3477f9 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:28:34.309575 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-23207569-dbbe-405b-ba61-28913d3477f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:28:34.390742 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1193/2375] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:28:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:28:34.400764 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8013b3b3-c8e0-4f58-a87a-9837bd4caeda admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:34.402383 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-8013b3b3-c8e0-4f58-a87a-9837bd4caeda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:28:34.478968 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8013b3b3-c8e0-4f58-a87a-9837bd4caeda admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:34.483528 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145254e6-6b1d-40d6-9222-c555622f0436 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:34.484513 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145254e6-6b1d-40d6-9222-c555622f0436 service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:34.484794 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145254e6-6b1d-40d6-9222-c555622f0436 service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:34.485066 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145254e6-6b1d-40d6-9222-c555622f0436 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:34.496929 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145254e6-6b1d-40d6-9222-c555622f0436 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:34.498377 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145254e6-6b1d-40d6-9222-c555622f0436 service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:34.508445 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1183/2376] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:28:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:28:35.016362 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23207569-dbbe-405b-ba61-28913d3477f9 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:35.018680 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-23207569-dbbe-405b-ba61-28913d3477f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:28:35.047727 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45223cd3-b706-4ca4-929d-a0e4e3d93f24 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:35.048497 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45223cd3-b706-4ca4-929d-a0e4e3d93f24 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:35.048497 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45223cd3-b706-4ca4-929d-a0e4e3d93f24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:35.050852 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23207569-dbbe-405b-ba61-28913d3477f9 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:35.062310 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bdb709-9f17-4f5c-ad66-dbed27503c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:35.062969 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bdb709-9f17-4f5c-ad66-dbed27503c62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:35.063246 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bdb709-9f17-4f5c-ad66-dbed27503c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:35.063550 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bdb709-9f17-4f5c-ad66-dbed27503c62 admin admin] PATH_INFO: `/v3/projects/92f71b8f614f4282b46d4cc3d03ff385` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:35.064691 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bdb709-9f17-4f5c-ad66-dbed27503c62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92f71b8f614f4282b46d4cc3d03ff385)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:35.067428 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bdb709-9f17-4f5c-ad66-dbed27503c62 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:35.072077 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-45223cd3-b706-4ca4-929d-a0e4e3d93f24 None None] MFA Rules not processed for user `d030d18b595144f19d6ad01d4cd5d52b`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:28:35.075930 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-45223cd3-b706-4ca4-929d-a0e4e3d93f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:28:35.106268 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:28:35.169592 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1184/2377] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Oct 12 00:28:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 00:28:35.193794 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45223cd3-b706-4ca4-929d-a0e4e3d93f24 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:35.195238 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-f9bdb709-9f17-4f5c-ad66-dbed27503c62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92f71b8f614f4282b46d4cc3d03ff385', 'request_id': 'req-f9bdb709-9f17-4f5c-ad66-dbed27503c62'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:35.196700 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-45223cd3-b706-4ca4-929d-a0e4e3d93f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:28:35.216981 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-f9bdb709-9f17-4f5c-ad66-dbed27503c62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92f71b8f614f4282b46d4cc3d03ff385', 'request_id': 'req-f9bdb709-9f17-4f5c-ad66-dbed27503c62'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:35.275902 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45223cd3-b706-4ca4-929d-a0e4e3d93f24 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:35.328414 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-f9bdb709-9f17-4f5c-ad66-dbed27503c62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92f71b8f614f4282b46d4cc3d03ff385', 'request_id': 'req-f9bdb709-9f17-4f5c-ad66-dbed27503c62'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:35.334681 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572424ba-4ce0-4084-a5a2-974099db6099 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc1bcffc78844dcbb1a9b7c80285249b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:35.335679 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572424ba-4ce0-4084-a5a2-974099db6099 service swift] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:35.336255 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572424ba-4ce0-4084-a5a2-974099db6099 service swift] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:35.336538 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572424ba-4ce0-4084-a5a2-974099db6099 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:35.339589 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-f9bdb709-9f17-4f5c-ad66-dbed27503c62 admin admin] The token cache is being invalidate because project 92f71b8f614f4282b46d4cc3d03ff385 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81747) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:28:35.339875 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-f9bdb709-9f17-4f5c-ad66-dbed27503c62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:35.340489 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-f9bdb709-9f17-4f5c-ad66-dbed27503c62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:35.341975 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1194/2378] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:28:35 2022] DELETE /identity/v3/projects/92f71b8f614f4282b46d4cc3d03ff385 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:28:35.345902 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bdb709-9f17-4f5c-ad66-dbed27503c62 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:35.347320 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-f9bdb709-9f17-4f5c-ad66-dbed27503c62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:28:35.399777 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572424ba-4ce0-4084-a5a2-974099db6099 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:35.401432 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572424ba-4ce0-4084-a5a2-974099db6099 service swift] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:35.409870 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1185/2379] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Oct 12 00:28:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 12 00:28:35.438249 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bdb709-9f17-4f5c-ad66-dbed27503c62 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:35.444217 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95482f3f-2763-4413-953a-041099e3929b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:35.444935 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95482f3f-2763-4413-953a-041099e3929b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:35.445377 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95482f3f-2763-4413-953a-041099e3929b admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:35.447610 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95482f3f-2763-4413-953a-041099e3929b admin admin] PATH_INFO: `/v3/users/f9374d64878a4ef7abf1267a406837bd` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:35.447610 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95482f3f-2763-4413-953a-041099e3929b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9374d64878a4ef7abf1267a406837bd)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:35.449029 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95482f3f-2763-4413-953a-041099e3929b admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:35.598321 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:28:35.700195 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-95482f3f-2763-4413-953a-041099e3929b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9374d64878a4ef7abf1267a406837bd'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:35.700875 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-95482f3f-2763-4413-953a-041099e3929b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9374d64878a4ef7abf1267a406837bd'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:35.713088 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-95482f3f-2763-4413-953a-041099e3929b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9374d64878a4ef7abf1267a406837bd'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:35.826219 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-95482f3f-2763-4413-953a-041099e3929b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9374d64878a4ef7abf1267a406837bd'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:35.827810 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-95482f3f-2763-4413-953a-041099e3929b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9374d64878a4ef7abf1267a406837bd'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:35.841129 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1195/2380] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:28:35 2022] DELETE /identity/v3/users/f9374d64878a4ef7abf1267a406837bd => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:28:36.247496 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572424ba-4ce0-4084-a5a2-974099db6099 service swift] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:36.256266 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:28:36.260891 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572424ba-4ce0-4084-a5a2-974099db6099 service swift] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:36.331121 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3e8065-3932-414d-91dc-b4655cc6eae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:36.331912 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e8065-3932-414d-91dc-b4655cc6eae4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:36.332287 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e8065-3932-414d-91dc-b4655cc6eae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:36.332555 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e8065-3932-414d-91dc-b4655cc6eae4 admin admin] PATH_INFO: `/v3/projects/1757acd523094d57b418c7a4a9b53227` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:36.333792 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3e8065-3932-414d-91dc-b4655cc6eae4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1757acd523094d57b418c7a4a9b53227)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:36.335709 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3e8065-3932-414d-91dc-b4655cc6eae4 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:36.360827 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa528fb-5227-4c2e-88e0-2e62516a34d8 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:36.361327 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa528fb-5227-4c2e-88e0-2e62516a34d8 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:36.361522 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa528fb-5227-4c2e-88e0-2e62516a34d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:36.381703 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-faa528fb-5227-4c2e-88e0-2e62516a34d8 None None] MFA Rules not processed for user `d030d18b595144f19d6ad01d4cd5d52b`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:28:36.382951 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-faa528fb-5227-4c2e-88e0-2e62516a34d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:28:36.442476 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1196/2381] 10.222.0.36 () {58 vars in 967 bytes} [Wed Oct 12 00:28:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 00:28:36.455601 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa528fb-5227-4c2e-88e0-2e62516a34d8 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:36.456968 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-faa528fb-5227-4c2e-88e0-2e62516a34d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:28:36.462262 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-ec3e8065-3932-414d-91dc-b4655cc6eae4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1757acd523094d57b418c7a4a9b53227', 'request_id': 'req-ec3e8065-3932-414d-91dc-b4655cc6eae4'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:36.470986 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-ec3e8065-3932-414d-91dc-b4655cc6eae4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1757acd523094d57b418c7a4a9b53227', 'request_id': 'req-ec3e8065-3932-414d-91dc-b4655cc6eae4'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:36.530177 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa528fb-5227-4c2e-88e0-2e62516a34d8 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:36.569261 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-ec3e8065-3932-414d-91dc-b4655cc6eae4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1757acd523094d57b418c7a4a9b53227', 'request_id': 'req-ec3e8065-3932-414d-91dc-b4655cc6eae4'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:36.579956 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:28:36.581979 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-ec3e8065-3932-414d-91dc-b4655cc6eae4 admin admin] The token cache is being invalidate because project 1757acd523094d57b418c7a4a9b53227 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81746) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:28:36.582378 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-ec3e8065-3932-414d-91dc-b4655cc6eae4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:36.583112 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-ec3e8065-3932-414d-91dc-b4655cc6eae4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:36.584740 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1186/2382] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:28:36 2022] DELETE /identity/v3/projects/1757acd523094d57b418c7a4a9b53227 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:28:36.590065 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c68420-97ea-4d6a-a27d-479c62d5ebb3 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:36.595630 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f0856a-df57-4706-af2c-09af9168662f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc1bcffc78844dcbb1a9b7c80285249b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:36.595946 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c68420-97ea-4d6a-a27d-479c62d5ebb3 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:36.597107 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f0856a-df57-4706-af2c-09af9168662f service swift] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:36.600756 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f0856a-df57-4706-af2c-09af9168662f service swift] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:36.601342 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c68420-97ea-4d6a-a27d-479c62d5ebb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:36.601515 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f0856a-df57-4706-af2c-09af9168662f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:36.601515 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-95f0856a-df57-4706-af2c-09af9168662f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:28:36.629493 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-b3c68420-97ea-4d6a-a27d-479c62d5ebb3 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:28:36.629493 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-b3c68420-97ea-4d6a-a27d-479c62d5ebb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:28:36.688298 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f0856a-df57-4706-af2c-09af9168662f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:36.690486 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f0856a-df57-4706-af2c-09af9168662f service swift] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:36.694804 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1187/2383] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:28:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 00:28:36.702818 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5273c33c-4a15-4650-904c-8191f172ffa1 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:36.703099 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5273c33c-4a15-4650-904c-8191f172ffa1 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:36.703393 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5273c33c-4a15-4650-904c-8191f172ffa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:36.706345 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1197/2384] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Oct 12 00:28:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 12 00:28:36.730862 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-5273c33c-4a15-4650-904c-8191f172ffa1 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:28:36.732509 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-5273c33c-4a15-4650-904c-8191f172ffa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:28:36.745482 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1188/2385] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:28:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 00:28:36.745606 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f0856a-df57-4706-af2c-09af9168662f service swift] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:36.756286 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f0856a-df57-4706-af2c-09af9168662f service swift] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:36.760860 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6fe506-a9fd-474b-bb4f-ed103e3ed227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:36.761655 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6fe506-a9fd-474b-bb4f-ed103e3ed227 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:36.761925 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6fe506-a9fd-474b-bb4f-ed103e3ed227 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:36.762214 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6fe506-a9fd-474b-bb4f-ed103e3ed227 admin admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:36.762828 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6fe506-a9fd-474b-bb4f-ed103e3ed227 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:28:36.763120 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6fe506-a9fd-474b-bb4f-ed103e3ed227 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:36.764739 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6fe506-a9fd-474b-bb4f-ed103e3ed227 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:36.777357 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1198/2386] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Oct 12 00:28:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:28:36.781799 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5273c33c-4a15-4650-904c-8191f172ffa1 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:36.791189 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5273c33c-4a15-4650-904c-8191f172ffa1 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:36.797080 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ad560f-9e9e-407e-a961-694533c92506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:36.797273 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad560f-9e9e-407e-a961-694533c92506 admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:36.797593 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad560f-9e9e-407e-a961-694533c92506 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:36.797831 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad560f-9e9e-407e-a961-694533c92506 admin admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:36.798460 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ad560f-9e9e-407e-a961-694533c92506 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:36.800434 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ad560f-9e9e-407e-a961-694533c92506 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:36.924824 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1189/2387] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:28:36 2022] POST /identity/v3/projects => generated 355 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:28:36.928935 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6fe506-a9fd-474b-bb4f-ed103e3ed227 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:36.938961 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6fe506-a9fd-474b-bb4f-ed103e3ed227 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:37.002718 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eac19ae-f5e6-410f-b23c-ba8f47a02d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:37.003323 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eac19ae-f5e6-410f-b23c-ba8f47a02d77 admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:37.003657 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eac19ae-f5e6-410f-b23c-ba8f47a02d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:37.003924 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eac19ae-f5e6-410f-b23c-ba8f47a02d77 admin admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:37.004556 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eac19ae-f5e6-410f-b23c-ba8f47a02d77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:37.006471 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eac19ae-f5e6-410f-b23c-ba8f47a02d77 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:37.084304 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1199/2388] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:28:36 2022] POST /identity/v3/users => generated 278 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 00:28:37.084525 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ad560f-9e9e-407e-a961-694533c92506 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:37.094881 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ad560f-9e9e-407e-a961-694533c92506 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:37.100050 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10640eea-dfb3-4a8c-8606-24f94efe6e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:37.100657 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10640eea-dfb3-4a8c-8606-24f94efe6e4e admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:37.101856 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10640eea-dfb3-4a8c-8606-24f94efe6e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:37.101856 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10640eea-dfb3-4a8c-8606-24f94efe6e4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:37.101856 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10640eea-dfb3-4a8c-8606-24f94efe6e4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:37.103418 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10640eea-dfb3-4a8c-8606-24f94efe6e4e admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:37.104889 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:28:37.114166 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1190/2389] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:28:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:28:37.118074 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eac19ae-f5e6-410f-b23c-ba8f47a02d77 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:37.130511 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eac19ae-f5e6-410f-b23c-ba8f47a02d77 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:37.135676 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961e34b0-ce3a-40ab-9437-80449e82d7b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:37.136829 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961e34b0-ce3a-40ab-9437-80449e82d7b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:37.137210 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961e34b0-ce3a-40ab-9437-80449e82d7b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:37.137474 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961e34b0-ce3a-40ab-9437-80449e82d7b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:37.138107 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961e34b0-ce3a-40ab-9437-80449e82d7b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:37.139854 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961e34b0-ce3a-40ab-9437-80449e82d7b5 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:37.154177 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1200/2390] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:28:37 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:28:37.163387 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10640eea-dfb3-4a8c-8606-24f94efe6e4e admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:37.175328 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10640eea-dfb3-4a8c-8606-24f94efe6e4e admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:37.183702 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666d0abc-c96a-4899-8841-e53b1e506aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:37.184463 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666d0abc-c96a-4899-8841-e53b1e506aef admin admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:37.184712 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666d0abc-c96a-4899-8841-e53b1e506aef admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:37.184981 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666d0abc-c96a-4899-8841-e53b1e506aef admin admin] PATH_INFO: `/v3/projects/7cf7661896894d07ba1e5fdfd5018392/users/07ad72d67e244330a617ff9183e12692/roles/ef689f96442945e8a7bc8cbfee13d909` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:37.201785 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666d0abc-c96a-4899-8841-e53b1e506aef admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cf7661896894d07ba1e5fdfd5018392, user_id=07ad72d67e244330a617ff9183e12692, role_id=ef689f96442945e8a7bc8cbfee13d909)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:37.204425 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666d0abc-c96a-4899-8841-e53b1e506aef admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:37.277363 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1191/2391] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:28:37 2022] PUT /identity/v3/projects/7cf7661896894d07ba1e5fdfd5018392/users/07ad72d67e244330a617ff9183e12692/roles/ef689f96442945e8a7bc8cbfee13d909 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:28:37.288967 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1716a4-851f-404c-a537-17933f7a3658 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:37.289113 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1716a4-851f-404c-a537-17933f7a3658 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:37.289383 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1716a4-851f-404c-a537-17933f7a3658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:37.294292 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0e85dd-5bf8-4d7a-be80-ce0f3ffde812 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:37.294765 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0e85dd-5bf8-4d7a-be80-ce0f3ffde812 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:37.295359 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0e85dd-5bf8-4d7a-be80-ce0f3ffde812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:37.315947 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-7e0e85dd-5bf8-4d7a-be80-ce0f3ffde812 None None] MFA Rules not processed for user `d030d18b595144f19d6ad01d4cd5d52b`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:28:37.317313 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-7e0e85dd-5bf8-4d7a-be80-ce0f3ffde812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:28:37.319244 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-8d1716a4-851f-404c-a537-17933f7a3658 None None] MFA Rules not processed for user `07ad72d67e244330a617ff9183e12692`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:28:37.320469 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-8d1716a4-851f-404c-a537-17933f7a3658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:28:37.374888 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:28:37.381583 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1192/2392] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Oct 12 00:28:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 00:28:37.403904 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0e85dd-5bf8-4d7a-be80-ce0f3ffde812 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:37.403904 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-7e0e85dd-5bf8-4d7a-be80-ce0f3ffde812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:28:37.404292 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1201/2393] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:28:37 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:28:37.409265 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1716a4-851f-404c-a537-17933f7a3658 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:37.470408 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0e85dd-5bf8-4d7a-be80-ce0f3ffde812 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:37.472137 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1716a4-851f-404c-a537-17933f7a3658 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:37.475664 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffb8979-a465-48b0-bb50-a9a98e4769fa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc1bcffc78844dcbb1a9b7c80285249b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:37.476274 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffb8979-a465-48b0-bb50-a9a98e4769fa service swift] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:37.476549 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffb8979-a465-48b0-bb50-a9a98e4769fa service swift] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:37.476817 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffb8979-a465-48b0-bb50-a9a98e4769fa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:37.477055 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c860e4-8e69-428f-8726-9cb5f1c2ddfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:37.477673 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c860e4-8e69-428f-8726-9cb5f1c2ddfd admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:37.477952 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c860e4-8e69-428f-8726-9cb5f1c2ddfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:37.478226 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c860e4-8e69-428f-8726-9cb5f1c2ddfd admin admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:37.478974 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c860e4-8e69-428f-8726-9cb5f1c2ddfd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:37.480996 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c860e4-8e69-428f-8726-9cb5f1c2ddfd admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:37.485356 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffb8979-a465-48b0-bb50-a9a98e4769fa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:37.486959 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffb8979-a465-48b0-bb50-a9a98e4769fa service swift] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:37.495265 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1193/2394] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Oct 12 00:28:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 12 00:28:37.569370 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1202/2395] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:28:37 2022] POST /identity/v3/projects => generated 357 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 00:28:37.573489 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffb8979-a465-48b0-bb50-a9a98e4769fa service swift] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:37.584666 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffb8979-a465-48b0-bb50-a9a98e4769fa service swift] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:37.643766 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d5ae78-bdfe-48ed-904c-1a2e909e60d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:37.644376 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d5ae78-bdfe-48ed-904c-1a2e909e60d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:37.644651 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d5ae78-bdfe-48ed-904c-1a2e909e60d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:37.644933 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d5ae78-bdfe-48ed-904c-1a2e909e60d1 admin admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:37.645542 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d5ae78-bdfe-48ed-904c-1a2e909e60d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:37.647546 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d5ae78-bdfe-48ed-904c-1a2e909e60d1 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:37.658552 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:28:37.755247 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1194/2396] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:28:37 2022] POST /identity/v3/users => generated 285 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:28:37.757256 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c860e4-8e69-428f-8726-9cb5f1c2ddfd admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:37.768227 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c860e4-8e69-428f-8726-9cb5f1c2ddfd admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:37.773135 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b01e4a6-3e10-411f-995f-01d251ed0f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:37.773733 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b01e4a6-3e10-411f-995f-01d251ed0f46 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:37.774007 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b01e4a6-3e10-411f-995f-01d251ed0f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:37.774275 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b01e4a6-3e10-411f-995f-01d251ed0f46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:37.774907 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b01e4a6-3e10-411f-995f-01d251ed0f46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:37.776538 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b01e4a6-3e10-411f-995f-01d251ed0f46 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:37.786430 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1203/2397] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:28:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:28:37.790572 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d5ae78-bdfe-48ed-904c-1a2e909e60d1 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:37.801180 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d5ae78-bdfe-48ed-904c-1a2e909e60d1 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:37.805805 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b23712-31ba-4e8d-8bbc-d4ef6d86ea86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:37.806456 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b23712-31ba-4e8d-8bbc-d4ef6d86ea86 admin admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:37.806749 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b23712-31ba-4e8d-8bbc-d4ef6d86ea86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:37.807022 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b23712-31ba-4e8d-8bbc-d4ef6d86ea86 admin admin] PATH_INFO: `/v3/projects/2245c8da15674b0f993c5785ff9e052a/users/a0f203d519814d519dba742db745156e/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:37.823274 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b23712-31ba-4e8d-8bbc-d4ef6d86ea86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2245c8da15674b0f993c5785ff9e052a, user_id=a0f203d519814d519dba742db745156e, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:37.826460 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b23712-31ba-4e8d-8bbc-d4ef6d86ea86 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:37.891971 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1195/2398] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:28:37 2022] PUT /identity/v3/projects/2245c8da15674b0f993c5785ff9e052a/users/a0f203d519814d519dba742db745156e/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:28:37.894640 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b01e4a6-3e10-411f-995f-01d251ed0f46 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:37.909106 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b01e4a6-3e10-411f-995f-01d251ed0f46 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:37.969758 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824b4f5f-dd9e-4318-ab48-1f6bbbbfe0cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:37.970356 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824b4f5f-dd9e-4318-ab48-1f6bbbbfe0cc admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:37.970628 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824b4f5f-dd9e-4318-ab48-1f6bbbbfe0cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:37.970933 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824b4f5f-dd9e-4318-ab48-1f6bbbbfe0cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:37.971628 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824b4f5f-dd9e-4318-ab48-1f6bbbbfe0cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:37.973237 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824b4f5f-dd9e-4318-ab48-1f6bbbbfe0cc admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:37.987206 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1204/2399] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:28:37 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:28:37.988337 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b23712-31ba-4e8d-8bbc-d4ef6d86ea86 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:38.000131 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b23712-31ba-4e8d-8bbc-d4ef6d86ea86 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:38.005609 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dd914c-eac6-4b49-8e71-0352a4789953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:38.006303 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dd914c-eac6-4b49-8e71-0352a4789953 admin admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:38.006600 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dd914c-eac6-4b49-8e71-0352a4789953 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:38.006831 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dd914c-eac6-4b49-8e71-0352a4789953 admin admin] PATH_INFO: `/v3/projects/2245c8da15674b0f993c5785ff9e052a/users/a0f203d519814d519dba742db745156e/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:38.013941 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dd914c-eac6-4b49-8e71-0352a4789953 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2245c8da15674b0f993c5785ff9e052a, user_id=a0f203d519814d519dba742db745156e, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:38.013941 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dd914c-eac6-4b49-8e71-0352a4789953 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:38.024393 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1196/2400] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:28:37 2022] PUT /identity/v3/projects/2245c8da15674b0f993c5785ff9e052a/users/a0f203d519814d519dba742db745156e/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 00:28:38.035646 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f438db-67ba-4f2a-a138-53c39ea4348f None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:38.035921 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f438db-67ba-4f2a-a138-53c39ea4348f None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:38.036194 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f438db-67ba-4f2a-a138-53c39ea4348f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:38.066400 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-98f438db-67ba-4f2a-a138-53c39ea4348f None None] MFA Rules not processed for user `a0f203d519814d519dba742db745156e`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:28:38.066543 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-98f438db-67ba-4f2a-a138-53c39ea4348f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:28:38.116668 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e9a891-a64e-4373-9cbf-03a5fad60cd3 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:38.122447 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e9a891-a64e-4373-9cbf-03a5fad60cd3 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:38.122447 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e9a891-a64e-4373-9cbf-03a5fad60cd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:38.124148 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:28:38.137660 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-69e9a891-a64e-4373-9cbf-03a5fad60cd3 None None] MFA Rules not processed for user `d030d18b595144f19d6ad01d4cd5d52b`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:28:38.138866 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-69e9a891-a64e-4373-9cbf-03a5fad60cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:28:38.149811 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1205/2401] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:28:38 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:28:38.160984 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f438db-67ba-4f2a-a138-53c39ea4348f None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:38.162381 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-98f438db-67ba-4f2a-a138-53c39ea4348f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:28:38.194639 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1197/2402] 10.222.0.36 () {58 vars in 967 bytes} [Wed Oct 12 00:28:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 00:28:38.208549 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e9a891-a64e-4373-9cbf-03a5fad60cd3 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:38.220053 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f438db-67ba-4f2a-a138-53c39ea4348f None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:38.223671 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a4d395-1730-4253-adbb-850c97d0c951 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:38.224274 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a4d395-1730-4253-adbb-850c97d0c951 service cinder] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:38.224549 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a4d395-1730-4253-adbb-850c97d0c951 service cinder] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:38.224817 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a4d395-1730-4253-adbb-850c97d0c951 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:38.234096 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a4d395-1730-4253-adbb-850c97d0c951 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:38.235631 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a4d395-1730-4253-adbb-850c97d0c951 service cinder] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:38.246035 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1206/2403] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:28:38 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 00:28:38.276297 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e9a891-a64e-4373-9cbf-03a5fad60cd3 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:38.281463 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49fc33e-def6-4240-bafd-ec64f62ffe42 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc1bcffc78844dcbb1a9b7c80285249b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:38.282060 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49fc33e-def6-4240-bafd-ec64f62ffe42 service swift] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:38.282334 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49fc33e-def6-4240-bafd-ec64f62ffe42 service swift] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:38.282603 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49fc33e-def6-4240-bafd-ec64f62ffe42 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:38.290644 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49fc33e-def6-4240-bafd-ec64f62ffe42 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:38.292286 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49fc33e-def6-4240-bafd-ec64f62ffe42 service swift] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:38.300370 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1198/2404] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Oct 12 00:28:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 12 00:28:38.439742 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a4d395-1730-4253-adbb-850c97d0c951 service cinder] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:38.441293 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-05a4d395-1730-4253-adbb-850c97d0c951 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:28:38.453346 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a4d395-1730-4253-adbb-850c97d0c951 service cinder] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:38.457061 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b81995-96ca-49e3-87d2-48c5a5b20671 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:38.457665 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b81995-96ca-49e3-87d2-48c5a5b20671 service cinder] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:38.457940 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b81995-96ca-49e3-87d2-48c5a5b20671 service cinder] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:38.458208 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b81995-96ca-49e3-87d2-48c5a5b20671 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:38.526909 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b81995-96ca-49e3-87d2-48c5a5b20671 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:38.527582 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b81995-96ca-49e3-87d2-48c5a5b20671 service cinder] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:38.588438 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1207/2405] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:28:38 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:28:39.991319 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49fc33e-def6-4240-bafd-ec64f62ffe42 service swift] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:39.992870 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-f49fc33e-def6-4240-bafd-ec64f62ffe42 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:28:40.058673 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49fc33e-def6-4240-bafd-ec64f62ffe42 service swift] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:40.058673 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8b54b8-c6b9-435e-8d63-b07b27327bc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:40.058673 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8b54b8-c6b9-435e-8d63-b07b27327bc1 service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:40.058673 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8b54b8-c6b9-435e-8d63-b07b27327bc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:40.059229 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8b54b8-c6b9-435e-8d63-b07b27327bc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:40.059229 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-aa8b54b8-c6b9-435e-8d63-b07b27327bc1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:28:40.064689 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:28:40.121459 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8b54b8-c6b9-435e-8d63-b07b27327bc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:40.122891 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8b54b8-c6b9-435e-8d63-b07b27327bc1 service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:40.132291 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1199/2406] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:28:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:28:41.127124 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b81995-96ca-49e3-87d2-48c5a5b20671 service cinder] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:41.128683 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-58b81995-96ca-49e3-87d2-48c5a5b20671 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:28:41.145382 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b81995-96ca-49e3-87d2-48c5a5b20671 service cinder] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:41.152196 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05b01c9-c609-4f47-8d5d-66e3e5d2f059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:41.152877 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05b01c9-c609-4f47-8d5d-66e3e5d2f059 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:41.153174 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05b01c9-c609-4f47-8d5d-66e3e5d2f059 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:41.153443 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05b01c9-c609-4f47-8d5d-66e3e5d2f059 admin admin] PATH_INFO: `/v3/users/0579e56a0b0b434389706f8f0b236ce3` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:41.154632 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05b01c9-c609-4f47-8d5d-66e3e5d2f059 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0579e56a0b0b434389706f8f0b236ce3)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:41.156618 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05b01c9-c609-4f47-8d5d-66e3e5d2f059 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:41.454328 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-b05b01c9-c609-4f47-8d5d-66e3e5d2f059 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0579e56a0b0b434389706f8f0b236ce3'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:41.455175 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-b05b01c9-c609-4f47-8d5d-66e3e5d2f059 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0579e56a0b0b434389706f8f0b236ce3'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:41.469255 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-b05b01c9-c609-4f47-8d5d-66e3e5d2f059 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0579e56a0b0b434389706f8f0b236ce3'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:41.551826 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-b05b01c9-c609-4f47-8d5d-66e3e5d2f059 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0579e56a0b0b434389706f8f0b236ce3'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:41.552680 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-b05b01c9-c609-4f47-8d5d-66e3e5d2f059 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0579e56a0b0b434389706f8f0b236ce3'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:41.561070 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:28:41.582008 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1208/2407] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:28:41 2022] DELETE /identity/v3/users/0579e56a0b0b434389706f8f0b236ce3 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:28:41.583974 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8f3795-acda-47f8-99c9-de640d98e9cb None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:41.584433 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8f3795-acda-47f8-99c9-de640d98e9cb None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:41.584871 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8f3795-acda-47f8-99c9-de640d98e9cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:41.608110 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-5a8f3795-acda-47f8-99c9-de640d98e9cb None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:28:41.610053 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-5a8f3795-acda-47f8-99c9-de640d98e9cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:28:41.763692 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1200/2408] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:28:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:28:41.778072 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05b01c9-c609-4f47-8d5d-66e3e5d2f059 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:41.780689 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-b05b01c9-c609-4f47-8d5d-66e3e5d2f059 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:28:41.871203 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05b01c9-c609-4f47-8d5d-66e3e5d2f059 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:41.876434 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b1b0eb-95fb-40cf-b2ae-32c67675d99f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:41.877066 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b1b0eb-95fb-40cf-b2ae-32c67675d99f service neutron] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:41.877339 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b1b0eb-95fb-40cf-b2ae-32c67675d99f service neutron] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:41.877608 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b1b0eb-95fb-40cf-b2ae-32c67675d99f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:41.894134 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b1b0eb-95fb-40cf-b2ae-32c67675d99f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:41.896609 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b1b0eb-95fb-40cf-b2ae-32c67675d99f service neutron] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:41.910312 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1209/2409] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:28:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:28:42.251974 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8f3795-acda-47f8-99c9-de640d98e9cb None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:42.253809 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-5a8f3795-acda-47f8-99c9-de640d98e9cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:28:42.276304 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8f3795-acda-47f8-99c9-de640d98e9cb None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:42.283815 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234ca513-192c-45d8-8cf7-f33458699695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:42.284474 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234ca513-192c-45d8-8cf7-f33458699695 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:42.285196 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234ca513-192c-45d8-8cf7-f33458699695 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:42.285196 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234ca513-192c-45d8-8cf7-f33458699695 admin admin] PATH_INFO: `/v3/projects/90c1e5e7db4942dea48cd03d186bb033` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:42.286473 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234ca513-192c-45d8-8cf7-f33458699695 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90c1e5e7db4942dea48cd03d186bb033)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:42.289192 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234ca513-192c-45d8-8cf7-f33458699695 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:42.468962 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-234ca513-192c-45d8-8cf7-f33458699695 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90c1e5e7db4942dea48cd03d186bb033', 'request_id': 'req-234ca513-192c-45d8-8cf7-f33458699695'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:42.527184 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-234ca513-192c-45d8-8cf7-f33458699695 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90c1e5e7db4942dea48cd03d186bb033', 'request_id': 'req-234ca513-192c-45d8-8cf7-f33458699695'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:42.597633 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-234ca513-192c-45d8-8cf7-f33458699695 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90c1e5e7db4942dea48cd03d186bb033', 'request_id': 'req-234ca513-192c-45d8-8cf7-f33458699695'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:42.608911 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-234ca513-192c-45d8-8cf7-f33458699695 admin admin] The token cache is being invalidate because project 90c1e5e7db4942dea48cd03d186bb033 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81746) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:28:42.609254 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-234ca513-192c-45d8-8cf7-f33458699695 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:42.609914 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-234ca513-192c-45d8-8cf7-f33458699695 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:42.611431 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1201/2410] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:28:42 2022] DELETE /identity/v3/projects/90c1e5e7db4942dea48cd03d186bb033 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:28:43.981621 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b1b0eb-95fb-40cf-b2ae-32c67675d99f service neutron] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:43.986196 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-59b1b0eb-95fb-40cf-b2ae-32c67675d99f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:28:43.997975 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b1b0eb-95fb-40cf-b2ae-32c67675d99f service neutron] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:44.063539 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf29aaa3-3a2c-4e4a-9027-ffbf4be3500f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:44.064180 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf29aaa3-3a2c-4e4a-9027-ffbf4be3500f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:44.064457 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf29aaa3-3a2c-4e4a-9027-ffbf4be3500f admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:44.064728 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf29aaa3-3a2c-4e4a-9027-ffbf4be3500f admin admin] PATH_INFO: `/v3/users/63474c55edbf4b22ad609471a2414ede` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:44.065864 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf29aaa3-3a2c-4e4a-9027-ffbf4be3500f admin admin] RBAC: Authorizing `identity:delete_user(user_id=63474c55edbf4b22ad609471a2414ede)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:44.067871 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf29aaa3-3a2c-4e4a-9027-ffbf4be3500f admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:44.234072 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-bf29aaa3-3a2c-4e4a-9027-ffbf4be3500f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63474c55edbf4b22ad609471a2414ede'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:44.234649 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-bf29aaa3-3a2c-4e4a-9027-ffbf4be3500f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63474c55edbf4b22ad609471a2414ede'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:44.248395 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-bf29aaa3-3a2c-4e4a-9027-ffbf4be3500f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63474c55edbf4b22ad609471a2414ede'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:44.299999 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-bf29aaa3-3a2c-4e4a-9027-ffbf4be3500f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63474c55edbf4b22ad609471a2414ede'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:44.300624 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-bf29aaa3-3a2c-4e4a-9027-ffbf4be3500f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63474c55edbf4b22ad609471a2414ede'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:44.314705 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1210/2411] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:28:43 2022] DELETE /identity/v3/users/63474c55edbf4b22ad609471a2414ede => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:28:44.541184 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234ca513-192c-45d8-8cf7-f33458699695 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:44.545711 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-234ca513-192c-45d8-8cf7-f33458699695 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:28:44.561050 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234ca513-192c-45d8-8cf7-f33458699695 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:44.585372 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:28:44.617315 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fb7365-370a-4b89-9c22-8149a3d9459d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:44.617978 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb7365-370a-4b89-9c22-8149a3d9459d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:44.618261 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb7365-370a-4b89-9c22-8149a3d9459d admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:44.618528 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb7365-370a-4b89-9c22-8149a3d9459d admin admin] PATH_INFO: `/v3/projects/13bdb0f5c58a40af9e420f6072f2f2b0` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:44.619670 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fb7365-370a-4b89-9c22-8149a3d9459d admin admin] RBAC: Authorizing `identity:delete_project(project_id=13bdb0f5c58a40af9e420f6072f2f2b0)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:44.622259 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fb7365-370a-4b89-9c22-8149a3d9459d admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:44.751626 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-90fb7365-370a-4b89-9c22-8149a3d9459d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13bdb0f5c58a40af9e420f6072f2f2b0', 'request_id': 'req-90fb7365-370a-4b89-9c22-8149a3d9459d'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:44.758405 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-90fb7365-370a-4b89-9c22-8149a3d9459d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13bdb0f5c58a40af9e420f6072f2f2b0', 'request_id': 'req-90fb7365-370a-4b89-9c22-8149a3d9459d'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:44.820277 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-90fb7365-370a-4b89-9c22-8149a3d9459d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13bdb0f5c58a40af9e420f6072f2f2b0', 'request_id': 'req-90fb7365-370a-4b89-9c22-8149a3d9459d'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:44.830986 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-90fb7365-370a-4b89-9c22-8149a3d9459d admin admin] The token cache is being invalidate because project 13bdb0f5c58a40af9e420f6072f2f2b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81746) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:28:44.831261 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-90fb7365-370a-4b89-9c22-8149a3d9459d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:44.832019 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-90fb7365-370a-4b89-9c22-8149a3d9459d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:44.833479 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1202/2412] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:28:44 2022] DELETE /identity/v3/projects/13bdb0f5c58a40af9e420f6072f2f2b0 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:28:44.838603 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6eb764-71c6-4312-8d80-36ffd6d18d54 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:44.838884 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6eb764-71c6-4312-8d80-36ffd6d18d54 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:44.839679 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6eb764-71c6-4312-8d80-36ffd6d18d54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:44.859749 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-6d6eb764-71c6-4312-8d80-36ffd6d18d54 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:28:44.860957 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-6d6eb764-71c6-4312-8d80-36ffd6d18d54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:28:44.872819 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:28:44.919385 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1211/2413] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:28:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:28:44.927195 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e55a3-37b1-420e-82ea-76324905d134 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:44.927552 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e55a3-37b1-420e-82ea-76324905d134 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:44.927894 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e55a3-37b1-420e-82ea-76324905d134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:44.947517 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-502e55a3-37b1-420e-82ea-76324905d134 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:28:44.948715 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-502e55a3-37b1-420e-82ea-76324905d134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:28:44.956772 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1203/2414] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:28:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:28:44.964808 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6eb764-71c6-4312-8d80-36ffd6d18d54 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:44.974299 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6eb764-71c6-4312-8d80-36ffd6d18d54 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:44.978835 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f00cb0-26c6-4406-bf46-58ae04cebe99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:44.980287 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f00cb0-26c6-4406-bf46-58ae04cebe99 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:44.984426 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f00cb0-26c6-4406-bf46-58ae04cebe99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:44.984426 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f00cb0-26c6-4406-bf46-58ae04cebe99 admin admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:44.984426 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f00cb0-26c6-4406-bf46-58ae04cebe99 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:28:44.984426 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f00cb0-26c6-4406-bf46-58ae04cebe99 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:44.984426 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f00cb0-26c6-4406-bf46-58ae04cebe99 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:44.995875 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1212/2415] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Oct 12 00:28:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:28:45.000064 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502e55a3-37b1-420e-82ea-76324905d134 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:45.009527 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502e55a3-37b1-420e-82ea-76324905d134 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:45.014350 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bd61a5-419b-426d-92cc-c06cf455287a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:45.014945 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bd61a5-419b-426d-92cc-c06cf455287a admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:45.015224 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bd61a5-419b-426d-92cc-c06cf455287a admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:45.015534 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bd61a5-419b-426d-92cc-c06cf455287a admin admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:45.016225 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bd61a5-419b-426d-92cc-c06cf455287a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:45.018128 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bd61a5-419b-426d-92cc-c06cf455287a admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:45.124646 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1204/2416] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:28:44 2022] POST /identity/v3/projects => generated 375 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:28:45.128843 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f00cb0-26c6-4406-bf46-58ae04cebe99 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:45.139760 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f00cb0-26c6-4406-bf46-58ae04cebe99 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:45.197466 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556d9be6-9c2b-4c5f-b8a8-969965d10e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:45.198096 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556d9be6-9c2b-4c5f-b8a8-969965d10e7c admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:45.198858 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556d9be6-9c2b-4c5f-b8a8-969965d10e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:45.199123 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556d9be6-9c2b-4c5f-b8a8-969965d10e7c admin admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:45.199824 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556d9be6-9c2b-4c5f-b8a8-969965d10e7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:45.201760 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556d9be6-9c2b-4c5f-b8a8-969965d10e7c admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:45.282527 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1213/2417] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:28:45 2022] POST /identity/v3/users => generated 288 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:28:45.286631 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bd61a5-419b-426d-92cc-c06cf455287a admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:45.297281 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bd61a5-419b-426d-92cc-c06cf455287a admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:45.302082 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4a88e8-e769-45a6-af80-b1cf216b7a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:45.302676 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4a88e8-e769-45a6-af80-b1cf216b7a5a admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:45.302948 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4a88e8-e769-45a6-af80-b1cf216b7a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:45.303213 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4a88e8-e769-45a6-af80-b1cf216b7a5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:45.303884 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4a88e8-e769-45a6-af80-b1cf216b7a5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:45.305465 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4a88e8-e769-45a6-af80-b1cf216b7a5a admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:45.315484 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1205/2418] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:28:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:28:45.319742 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556d9be6-9c2b-4c5f-b8a8-969965d10e7c admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:45.330278 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556d9be6-9c2b-4c5f-b8a8-969965d10e7c admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:45.334921 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ce6427-18e4-4960-8852-ae17f09ab1ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:45.335559 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce6427-18e4-4960-8852-ae17f09ab1ea admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:45.339666 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce6427-18e4-4960-8852-ae17f09ab1ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:45.339666 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce6427-18e4-4960-8852-ae17f09ab1ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:45.339666 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ce6427-18e4-4960-8852-ae17f09ab1ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:45.339666 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ce6427-18e4-4960-8852-ae17f09ab1ea admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:45.355158 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1214/2419] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:28:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:28:45.358893 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4a88e8-e769-45a6-af80-b1cf216b7a5a admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:45.369117 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4a88e8-e769-45a6-af80-b1cf216b7a5a admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:45.373798 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fa33f5-da6f-4cac-b7a5-5ff9cdb19242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:45.374450 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fa33f5-da6f-4cac-b7a5-5ff9cdb19242 admin admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:45.374780 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fa33f5-da6f-4cac-b7a5-5ff9cdb19242 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:45.375000 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fa33f5-da6f-4cac-b7a5-5ff9cdb19242 admin admin] PATH_INFO: `/v3/projects/2366d2cda6374336ad68b9e67220f620/users/0164e18368ca40828ce322f396bc5999/roles/ef689f96442945e8a7bc8cbfee13d909` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:45.391214 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fa33f5-da6f-4cac-b7a5-5ff9cdb19242 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2366d2cda6374336ad68b9e67220f620, user_id=0164e18368ca40828ce322f396bc5999, role_id=ef689f96442945e8a7bc8cbfee13d909)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:45.394284 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fa33f5-da6f-4cac-b7a5-5ff9cdb19242 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:45.488330 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1206/2420] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:28:45 2022] PUT /identity/v3/projects/2366d2cda6374336ad68b9e67220f620/users/0164e18368ca40828ce322f396bc5999/roles/ef689f96442945e8a7bc8cbfee13d909 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:28:45.500419 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aacf7a-4fba-4c55-97a3-e569edc863c5 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:45.500702 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aacf7a-4fba-4c55-97a3-e569edc863c5 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:45.500974 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aacf7a-4fba-4c55-97a3-e569edc863c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:45.531980 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-50aacf7a-4fba-4c55-97a3-e569edc863c5 None None] MFA Rules not processed for user `0164e18368ca40828ce322f396bc5999`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:28:45.533184 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-50aacf7a-4fba-4c55-97a3-e569edc863c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:28:45.605361 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1215/2421] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:28:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:28:45.616836 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fa33f5-da6f-4cac-b7a5-5ff9cdb19242 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:45.618651 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-76fa33f5-da6f-4cac-b7a5-5ff9cdb19242 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:28:45.676896 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fa33f5-da6f-4cac-b7a5-5ff9cdb19242 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:45.680614 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bb1ef2-2939-41f1-ab66-e63175489d0f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59b883abbb674b3c976bacb64194a66d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:45.681243 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bb1ef2-2939-41f1-ab66-e63175489d0f service glance] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:45.681523 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bb1ef2-2939-41f1-ab66-e63175489d0f service glance] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:45.681793 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bb1ef2-2939-41f1-ab66-e63175489d0f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:45.688528 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:28:45.690687 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bb1ef2-2939-41f1-ab66-e63175489d0f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:45.692274 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bb1ef2-2939-41f1-ab66-e63175489d0f service glance] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:45.701744 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1207/2422] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:28:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:28:55.602375 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=91,keystone:INSERT=3 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:28:56.007755 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=2 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:28:57.474656 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aacf7a-4fba-4c55-97a3-e569edc863c5 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:57.476092 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-50aacf7a-4fba-4c55-97a3-e569edc863c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:28:57.544357 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aacf7a-4fba-4c55-97a3-e569edc863c5 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:57.549381 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e960143-1f31-4931-b5cf-5fc2be8785f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:57.550043 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e960143-1f31-4931-b5cf-5fc2be8785f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:57.550313 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e960143-1f31-4931-b5cf-5fc2be8785f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:57.550578 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e960143-1f31-4931-b5cf-5fc2be8785f5 admin admin] PATH_INFO: `/v3/users/07ad72d67e244330a617ff9183e12692` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:57.551709 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e960143-1f31-4931-b5cf-5fc2be8785f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07ad72d67e244330a617ff9183e12692)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:57.553611 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e960143-1f31-4931-b5cf-5fc2be8785f5 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:57.725857 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-0e960143-1f31-4931-b5cf-5fc2be8785f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07ad72d67e244330a617ff9183e12692'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:57.726585 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-0e960143-1f31-4931-b5cf-5fc2be8785f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07ad72d67e244330a617ff9183e12692'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:57.739146 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-0e960143-1f31-4931-b5cf-5fc2be8785f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07ad72d67e244330a617ff9183e12692'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:57.803083 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-0e960143-1f31-4931-b5cf-5fc2be8785f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07ad72d67e244330a617ff9183e12692'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:57.803733 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-0e960143-1f31-4931-b5cf-5fc2be8785f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07ad72d67e244330a617ff9183e12692'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:57.817008 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1216/2423] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:28:57 2022] DELETE /identity/v3/users/07ad72d67e244330a617ff9183e12692 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:28:57.821548 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c98a0ed-8410-4348-a410-4f91795223e8 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:57.822047 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c98a0ed-8410-4348-a410-4f91795223e8 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:57.822198 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c98a0ed-8410-4348-a410-4f91795223e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:57.840989 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-6c98a0ed-8410-4348-a410-4f91795223e8 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:28:57.842175 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-6c98a0ed-8410-4348-a410-4f91795223e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:28:57.902277 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1208/2424] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:28:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:28:57.912715 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e960143-1f31-4931-b5cf-5fc2be8785f5 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:57.914139 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-0e960143-1f31-4931-b5cf-5fc2be8785f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:28:57.971982 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e960143-1f31-4931-b5cf-5fc2be8785f5 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:57.975844 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80a768e-95a0-45da-8a76-e30e0eaedec6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:57.976454 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80a768e-95a0-45da-8a76-e30e0eaedec6 service neutron] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:57.976727 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80a768e-95a0-45da-8a76-e30e0eaedec6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:57.976998 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80a768e-95a0-45da-8a76-e30e0eaedec6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:57.986020 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80a768e-95a0-45da-8a76-e30e0eaedec6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:57.987472 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80a768e-95a0-45da-8a76-e30e0eaedec6 service neutron] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:57.997249 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1217/2425] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:28:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:28:58.438680 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c98a0ed-8410-4348-a410-4f91795223e8 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:58.440206 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-6c98a0ed-8410-4348-a410-4f91795223e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:28:58.455925 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c98a0ed-8410-4348-a410-4f91795223e8 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:58.460660 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4fb2d7-a0e8-4614-bb19-7d79d9cbe65b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:58.461303 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4fb2d7-a0e8-4614-bb19-7d79d9cbe65b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:58.461584 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4fb2d7-a0e8-4614-bb19-7d79d9cbe65b admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:58.461875 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4fb2d7-a0e8-4614-bb19-7d79d9cbe65b admin admin] PATH_INFO: `/v3/projects/7cf7661896894d07ba1e5fdfd5018392` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:58.463073 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4fb2d7-a0e8-4614-bb19-7d79d9cbe65b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7cf7661896894d07ba1e5fdfd5018392)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:58.468812 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4fb2d7-a0e8-4614-bb19-7d79d9cbe65b admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:58.574621 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-7b4fb2d7-a0e8-4614-bb19-7d79d9cbe65b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7cf7661896894d07ba1e5fdfd5018392', 'request_id': 'req-7b4fb2d7-a0e8-4614-bb19-7d79d9cbe65b'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:58.580975 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-7b4fb2d7-a0e8-4614-bb19-7d79d9cbe65b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7cf7661896894d07ba1e5fdfd5018392', 'request_id': 'req-7b4fb2d7-a0e8-4614-bb19-7d79d9cbe65b'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:58.625301 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-7b4fb2d7-a0e8-4614-bb19-7d79d9cbe65b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7cf7661896894d07ba1e5fdfd5018392', 'request_id': 'req-7b4fb2d7-a0e8-4614-bb19-7d79d9cbe65b'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:58.635484 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-7b4fb2d7-a0e8-4614-bb19-7d79d9cbe65b admin admin] The token cache is being invalidate because project 7cf7661896894d07ba1e5fdfd5018392 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81746) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:28:58.635773 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-7b4fb2d7-a0e8-4614-bb19-7d79d9cbe65b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:58.636567 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-7b4fb2d7-a0e8-4614-bb19-7d79d9cbe65b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:58.638090 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1209/2426] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:28:58 2022] DELETE /identity/v3/projects/7cf7661896894d07ba1e5fdfd5018392 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:28:58.642198 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80a768e-95a0-45da-8a76-e30e0eaedec6 service neutron] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:58.652118 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80a768e-95a0-45da-8a76-e30e0eaedec6 service neutron] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:58.669564 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:28:58.707258 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aa9aee-32c3-4152-9b3f-4416644cb79b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:58.707912 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aa9aee-32c3-4152-9b3f-4416644cb79b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:58.708183 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aa9aee-32c3-4152-9b3f-4416644cb79b admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:58.708446 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aa9aee-32c3-4152-9b3f-4416644cb79b admin admin] PATH_INFO: `/v3/users/a0f203d519814d519dba742db745156e` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:58.709623 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aa9aee-32c3-4152-9b3f-4416644cb79b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0f203d519814d519dba742db745156e)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:58.711513 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aa9aee-32c3-4152-9b3f-4416644cb79b admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:58.816301 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-a9aa9aee-32c3-4152-9b3f-4416644cb79b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0f203d519814d519dba742db745156e'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:58.816960 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-a9aa9aee-32c3-4152-9b3f-4416644cb79b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0f203d519814d519dba742db745156e'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:58.828730 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-a9aa9aee-32c3-4152-9b3f-4416644cb79b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0f203d519814d519dba742db745156e'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:58.916064 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-a9aa9aee-32c3-4152-9b3f-4416644cb79b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0f203d519814d519dba742db745156e'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:58.916678 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-a9aa9aee-32c3-4152-9b3f-4416644cb79b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0f203d519814d519dba742db745156e'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:58.930048 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1218/2427] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:28:58 2022] DELETE /identity/v3/users/a0f203d519814d519dba742db745156e => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:28:59.542339 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4fb2d7-a0e8-4614-bb19-7d79d9cbe65b admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:59.544011 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-7b4fb2d7-a0e8-4614-bb19-7d79d9cbe65b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:28:59.557420 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4fb2d7-a0e8-4614-bb19-7d79d9cbe65b admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:59.614149 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4a972c-4b90-4bbf-b8d6-b4027fc1039c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:59.614713 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4a972c-4b90-4bbf-b8d6-b4027fc1039c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:59.614991 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4a972c-4b90-4bbf-b8d6-b4027fc1039c admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:59.615263 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4a972c-4b90-4bbf-b8d6-b4027fc1039c admin admin] PATH_INFO: `/v3/projects/2245c8da15674b0f993c5785ff9e052a` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:59.616407 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4a972c-4b90-4bbf-b8d6-b4027fc1039c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2245c8da15674b0f993c5785ff9e052a)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:59.618311 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4a972c-4b90-4bbf-b8d6-b4027fc1039c admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:59.712005 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:28:59.727801 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-ae4a972c-4b90-4bbf-b8d6-b4027fc1039c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2245c8da15674b0f993c5785ff9e052a', 'request_id': 'req-ae4a972c-4b90-4bbf-b8d6-b4027fc1039c'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:59.734005 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-ae4a972c-4b90-4bbf-b8d6-b4027fc1039c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2245c8da15674b0f993c5785ff9e052a', 'request_id': 'req-ae4a972c-4b90-4bbf-b8d6-b4027fc1039c'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:59.802719 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-ae4a972c-4b90-4bbf-b8d6-b4027fc1039c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2245c8da15674b0f993c5785ff9e052a', 'request_id': 'req-ae4a972c-4b90-4bbf-b8d6-b4027fc1039c'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:59.814556 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-ae4a972c-4b90-4bbf-b8d6-b4027fc1039c admin admin] The token cache is being invalidate because project 2245c8da15674b0f993c5785ff9e052a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81746) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:28:59.815426 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-ae4a972c-4b90-4bbf-b8d6-b4027fc1039c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:59.816074 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-ae4a972c-4b90-4bbf-b8d6-b4027fc1039c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:28:59.817589 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1210/2428] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:28:59 2022] DELETE /identity/v3/projects/2245c8da15674b0f993c5785ff9e052a => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:28:59.822441 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d475111e-4a41-4bd8-b254-a7021227f8d3 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:59.822909 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d475111e-4a41-4bd8-b254-a7021227f8d3 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:59.823007 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d475111e-4a41-4bd8-b254-a7021227f8d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:59.843203 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-d475111e-4a41-4bd8-b254-a7021227f8d3 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:28:59.844454 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-d475111e-4a41-4bd8-b254-a7021227f8d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:28:59.902455 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1219/2429] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:28:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:28:59.910491 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41cab58-07f1-40f3-acea-e229ac79db9c None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:59.910775 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41cab58-07f1-40f3-acea-e229ac79db9c None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:59.911045 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41cab58-07f1-40f3-acea-e229ac79db9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:59.930765 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-b41cab58-07f1-40f3-acea-e229ac79db9c None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:28:59.931986 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-b41cab58-07f1-40f3-acea-e229ac79db9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:28:59.940320 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1211/2430] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:28:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:28:59.944356 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d475111e-4a41-4bd8-b254-a7021227f8d3 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:59.954455 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d475111e-4a41-4bd8-b254-a7021227f8d3 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:59.959165 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5915add0-3015-4cc6-bc1e-7001d489b2a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:59.959812 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5915add0-3015-4cc6-bc1e-7001d489b2a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:59.960090 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5915add0-3015-4cc6-bc1e-7001d489b2a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:59.960411 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5915add0-3015-4cc6-bc1e-7001d489b2a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:59.961078 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5915add0-3015-4cc6-bc1e-7001d489b2a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:28:59.961378 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5915add0-3015-4cc6-bc1e-7001d489b2a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:59.962978 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5915add0-3015-4cc6-bc1e-7001d489b2a1 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:28:59.974634 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1220/2431] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Oct 12 00:28:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:28:59.978510 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41cab58-07f1-40f3-acea-e229ac79db9c None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:28:59.987981 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41cab58-07f1-40f3-acea-e229ac79db9c None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:28:59.992837 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae0d85c-5cfd-47cf-9402-cbe901644b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:28:59.993430 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae0d85c-5cfd-47cf-9402-cbe901644b7e admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:28:59.993712 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae0d85c-5cfd-47cf-9402-cbe901644b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:28:59.993987 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae0d85c-5cfd-47cf-9402-cbe901644b7e admin admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:28:59.994629 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae0d85c-5cfd-47cf-9402-cbe901644b7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:28:59.996626 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae0d85c-5cfd-47cf-9402-cbe901644b7e admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:29:00.089828 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1212/2432] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:28:59 2022] POST /identity/v3/projects => generated 387 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:29:00.094045 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5915add0-3015-4cc6-bc1e-7001d489b2a1 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:29:00.104158 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5915add0-3015-4cc6-bc1e-7001d489b2a1 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:29:00.131669 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:29:00.164402 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96b900f-2ca2-4db4-b23b-c09c1912aec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:29:00.164939 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96b900f-2ca2-4db4-b23b-c09c1912aec0 admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:00.165240 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96b900f-2ca2-4db4-b23b-c09c1912aec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:00.165508 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96b900f-2ca2-4db4-b23b-c09c1912aec0 admin admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:00.166140 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96b900f-2ca2-4db4-b23b-c09c1912aec0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:29:00.168074 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96b900f-2ca2-4db4-b23b-c09c1912aec0 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:29:00.230936 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1221/2433] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:29:00 2022] POST /identity/v3/users => generated 294 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 00:29:00.235642 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae0d85c-5cfd-47cf-9402-cbe901644b7e admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:29:00.246148 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae0d85c-5cfd-47cf-9402-cbe901644b7e admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:29:00.251550 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8768d8cb-985b-4094-91de-4c46e87bf694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:29:00.252155 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8768d8cb-985b-4094-91de-4c46e87bf694 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:00.252490 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8768d8cb-985b-4094-91de-4c46e87bf694 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:00.252755 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8768d8cb-985b-4094-91de-4c46e87bf694 admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:00.253429 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8768d8cb-985b-4094-91de-4c46e87bf694 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:29:00.255027 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8768d8cb-985b-4094-91de-4c46e87bf694 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:29:00.265075 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1213/2434] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:29:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:29:00.269431 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96b900f-2ca2-4db4-b23b-c09c1912aec0 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:29:00.279310 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96b900f-2ca2-4db4-b23b-c09c1912aec0 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:29:00.283944 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ba8ba5-4285-4129-b28f-3f8b126dbe99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:29:00.284526 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba8ba5-4285-4129-b28f-3f8b126dbe99 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:00.284798 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba8ba5-4285-4129-b28f-3f8b126dbe99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:00.285066 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba8ba5-4285-4129-b28f-3f8b126dbe99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:00.285696 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ba8ba5-4285-4129-b28f-3f8b126dbe99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:29:00.287264 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ba8ba5-4285-4129-b28f-3f8b126dbe99 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:29:00.297691 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1222/2435] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:29:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:29:00.301663 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8768d8cb-985b-4094-91de-4c46e87bf694 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:29:00.311583 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8768d8cb-985b-4094-91de-4c46e87bf694 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:29:00.316789 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa01eb53-e838-41d1-89b7-de80aedfe90b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:29:00.317446 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa01eb53-e838-41d1-89b7-de80aedfe90b admin admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:00.317726 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa01eb53-e838-41d1-89b7-de80aedfe90b admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:00.318000 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa01eb53-e838-41d1-89b7-de80aedfe90b admin admin] PATH_INFO: `/v3/projects/4f3b24ad523f4bc0afa901454cfbb69a/users/2d69da321706436780b94e0401456204/roles/ef689f96442945e8a7bc8cbfee13d909` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:00.332120 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa01eb53-e838-41d1-89b7-de80aedfe90b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f3b24ad523f4bc0afa901454cfbb69a, user_id=2d69da321706436780b94e0401456204, role_id=ef689f96442945e8a7bc8cbfee13d909)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:29:00.334756 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa01eb53-e838-41d1-89b7-de80aedfe90b admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:29:00.380005 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1214/2436] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:29:00 2022] PUT /identity/v3/projects/4f3b24ad523f4bc0afa901454cfbb69a/users/2d69da321706436780b94e0401456204/roles/ef689f96442945e8a7bc8cbfee13d909 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:29:00.447673 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb77fdb-0992-4d88-a7d5-0aeb53fb7c1e None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:00.447901 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb77fdb-0992-4d88-a7d5-0aeb53fb7c1e None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:00.448116 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb77fdb-0992-4d88-a7d5-0aeb53fb7c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:00.476488 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-ebb77fdb-0992-4d88-a7d5-0aeb53fb7c1e None None] MFA Rules not processed for user `2d69da321706436780b94e0401456204`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:29:00.477692 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-ebb77fdb-0992-4d88-a7d5-0aeb53fb7c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:29:00.539418 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1223/2437] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:29:00 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:29:00.548037 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa01eb53-e838-41d1-89b7-de80aedfe90b admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:29:00.610060 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa01eb53-e838-41d1-89b7-de80aedfe90b admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:29:00.614793 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6785a8-65f9-47d2-8c30-f3c5f752ed98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:29:00.615449 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6785a8-65f9-47d2-8c30-f3c5f752ed98 admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:00.615729 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6785a8-65f9-47d2-8c30-f3c5f752ed98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:00.616031 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6785a8-65f9-47d2-8c30-f3c5f752ed98 admin admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:00.616658 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6785a8-65f9-47d2-8c30-f3c5f752ed98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:29:00.618559 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6785a8-65f9-47d2-8c30-f3c5f752ed98 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:29:00.701036 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1215/2438] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:29:00 2022] POST /identity/v3/projects => generated 385 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:29:00.705111 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb77fdb-0992-4d88-a7d5-0aeb53fb7c1e None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:29:00.715691 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb77fdb-0992-4d88-a7d5-0aeb53fb7c1e None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:29:00.772145 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e779e1e2-aa1e-4f2e-858d-fc7ed2a67fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:29:00.772746 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e779e1e2-aa1e-4f2e-858d-fc7ed2a67fdc admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:00.773019 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e779e1e2-aa1e-4f2e-858d-fc7ed2a67fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:00.773284 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e779e1e2-aa1e-4f2e-858d-fc7ed2a67fdc admin admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:00.773907 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e779e1e2-aa1e-4f2e-858d-fc7ed2a67fdc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:29:00.775819 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e779e1e2-aa1e-4f2e-858d-fc7ed2a67fdc admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:29:00.836882 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1224/2439] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:29:00 2022] POST /identity/v3/users => generated 299 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:29:00.841409 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6785a8-65f9-47d2-8c30-f3c5f752ed98 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:29:00.852442 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6785a8-65f9-47d2-8c30-f3c5f752ed98 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:29:00.857091 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f4bc6e-b6e7-4c5c-bc9d-d744650aaf0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:29:00.857679 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f4bc6e-b6e7-4c5c-bc9d-d744650aaf0d admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:00.857956 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f4bc6e-b6e7-4c5c-bc9d-d744650aaf0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:00.858228 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f4bc6e-b6e7-4c5c-bc9d-d744650aaf0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:00.858864 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f4bc6e-b6e7-4c5c-bc9d-d744650aaf0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:29:00.860484 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f4bc6e-b6e7-4c5c-bc9d-d744650aaf0d admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:29:00.871861 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1216/2440] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:29:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:29:00.875912 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e779e1e2-aa1e-4f2e-858d-fc7ed2a67fdc admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:29:00.885788 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e779e1e2-aa1e-4f2e-858d-fc7ed2a67fdc admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:29:00.890921 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bf91fc-3adf-488d-9f0e-a9a119fea0b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:29:00.891703 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bf91fc-3adf-488d-9f0e-a9a119fea0b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:00.892085 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bf91fc-3adf-488d-9f0e-a9a119fea0b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:00.892531 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bf91fc-3adf-488d-9f0e-a9a119fea0b0 admin admin] PATH_INFO: `/v3/projects/ced59da82b9945638f9f5ebb447eb30d/users/1a1764046a324cac81150aecca87bc10/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:00.907843 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bf91fc-3adf-488d-9f0e-a9a119fea0b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ced59da82b9945638f9f5ebb447eb30d, user_id=1a1764046a324cac81150aecca87bc10, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:29:00.910444 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bf91fc-3adf-488d-9f0e-a9a119fea0b0 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:29:00.966862 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1225/2441] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:29:00 2022] PUT /identity/v3/projects/ced59da82b9945638f9f5ebb447eb30d/users/1a1764046a324cac81150aecca87bc10/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:29:00.971338 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f4bc6e-b6e7-4c5c-bc9d-d744650aaf0d admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:29:00.982201 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f4bc6e-b6e7-4c5c-bc9d-d744650aaf0d admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:29:01.001505 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:29:01.037742 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c97a08-40ca-4324-b4d8-4a1eab31e91f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:29:01.038350 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c97a08-40ca-4324-b4d8-4a1eab31e91f admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:01.038624 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c97a08-40ca-4324-b4d8-4a1eab31e91f admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:01.038895 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c97a08-40ca-4324-b4d8-4a1eab31e91f admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:01.039559 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c97a08-40ca-4324-b4d8-4a1eab31e91f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:29:01.041148 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c97a08-40ca-4324-b4d8-4a1eab31e91f admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:29:01.050621 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1217/2442] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:29:00 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:29:01.054889 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bf91fc-3adf-488d-9f0e-a9a119fea0b0 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:29:01.060193 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:29:01.065349 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bf91fc-3adf-488d-9f0e-a9a119fea0b0 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:29:01.069980 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95b9555-689a-4a57-a459-2ac7b7666ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:29:01.074284 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95b9555-689a-4a57-a459-2ac7b7666ec5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:01.074284 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95b9555-689a-4a57-a459-2ac7b7666ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:01.074284 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95b9555-689a-4a57-a459-2ac7b7666ec5 admin admin] PATH_INFO: `/v3/projects/ced59da82b9945638f9f5ebb447eb30d/users/1a1764046a324cac81150aecca87bc10/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:01.074284 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95b9555-689a-4a57-a459-2ac7b7666ec5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ced59da82b9945638f9f5ebb447eb30d, user_id=1a1764046a324cac81150aecca87bc10, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:29:01.075705 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95b9555-689a-4a57-a459-2ac7b7666ec5 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:29:01.086368 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1226/2443] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:29:01 2022] PUT /identity/v3/projects/ced59da82b9945638f9f5ebb447eb30d/users/1a1764046a324cac81150aecca87bc10/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 00:29:01.096771 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35383f8c-e589-4f04-bbe7-5ec6eca4e730 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:01.097079 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35383f8c-e589-4f04-bbe7-5ec6eca4e730 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:01.097358 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35383f8c-e589-4f04-bbe7-5ec6eca4e730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:01.124300 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-35383f8c-e589-4f04-bbe7-5ec6eca4e730 None None] MFA Rules not processed for user `1a1764046a324cac81150aecca87bc10`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:29:01.125507 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-35383f8c-e589-4f04-bbe7-5ec6eca4e730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:29:01.192057 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1218/2444] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:29:01 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:29:01.203178 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95b9555-689a-4a57-a459-2ac7b7666ec5 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:29:01.204970 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-e95b9555-689a-4a57-a459-2ac7b7666ec5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:29:01.261988 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95b9555-689a-4a57-a459-2ac7b7666ec5 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:29:01.266239 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb03ee42-48f3-41f9-880d-0dcad8652566 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:29:01.266806 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb03ee42-48f3-41f9-880d-0dcad8652566 service cinder] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:01.267086 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb03ee42-48f3-41f9-880d-0dcad8652566 service cinder] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:01.267350 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb03ee42-48f3-41f9-880d-0dcad8652566 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:01.276724 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb03ee42-48f3-41f9-880d-0dcad8652566 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:29:01.278148 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb03ee42-48f3-41f9-880d-0dcad8652566 service cinder] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:29:01.287598 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1227/2445] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:29:01 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:29:02.315082 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35383f8c-e589-4f04-bbe7-5ec6eca4e730 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:29:02.376599 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35383f8c-e589-4f04-bbe7-5ec6eca4e730 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:29:02.381147 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251bf3e3-dd1b-453c-9a96-66809fd23d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:29:02.381772 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251bf3e3-dd1b-453c-9a96-66809fd23d81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:02.382047 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251bf3e3-dd1b-453c-9a96-66809fd23d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:02.382316 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251bf3e3-dd1b-453c-9a96-66809fd23d81 admin admin] PATH_INFO: `/v3/users/2d69da321706436780b94e0401456204` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:02.383508 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251bf3e3-dd1b-453c-9a96-66809fd23d81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d69da321706436780b94e0401456204)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:29:02.385392 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251bf3e3-dd1b-453c-9a96-66809fd23d81 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:29:02.616051 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-251bf3e3-dd1b-453c-9a96-66809fd23d81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d69da321706436780b94e0401456204'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:29:02.616698 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-251bf3e3-dd1b-453c-9a96-66809fd23d81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d69da321706436780b94e0401456204'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:29:02.617883 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:29:02.629346 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-251bf3e3-dd1b-453c-9a96-66809fd23d81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d69da321706436780b94e0401456204'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:29:02.688622 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-251bf3e3-dd1b-453c-9a96-66809fd23d81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d69da321706436780b94e0401456204'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:29:02.689223 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-251bf3e3-dd1b-453c-9a96-66809fd23d81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d69da321706436780b94e0401456204'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:29:02.701957 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1219/2446] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:29:02 2022] DELETE /identity/v3/users/2d69da321706436780b94e0401456204 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:29:02.707000 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7b8fb0-74ef-43c1-88b5-507742cd9b4a None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:02.707492 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7b8fb0-74ef-43c1-88b5-507742cd9b4a None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:02.707649 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7b8fb0-74ef-43c1-88b5-507742cd9b4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:02.726917 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-ec7b8fb0-74ef-43c1-88b5-507742cd9b4a None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:29:02.728152 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-ec7b8fb0-74ef-43c1-88b5-507742cd9b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:29:02.786144 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1228/2447] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:29:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:29:02.796089 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251bf3e3-dd1b-453c-9a96-66809fd23d81 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:29:02.797519 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-251bf3e3-dd1b-453c-9a96-66809fd23d81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:29:02.854207 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251bf3e3-dd1b-453c-9a96-66809fd23d81 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:29:02.857801 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fc5c86-5ff5-45e2-aaeb-bece47f08325 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:29:02.858441 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fc5c86-5ff5-45e2-aaeb-bece47f08325 service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:02.858721 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fc5c86-5ff5-45e2-aaeb-bece47f08325 service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:02.858992 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fc5c86-5ff5-45e2-aaeb-bece47f08325 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:02.868018 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fc5c86-5ff5-45e2-aaeb-bece47f08325 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:29:02.869453 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fc5c86-5ff5-45e2-aaeb-bece47f08325 service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:29:02.879471 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1220/2448] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:29:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:29:03.289851 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7b8fb0-74ef-43c1-88b5-507742cd9b4a None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:29:03.291397 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-ec7b8fb0-74ef-43c1-88b5-507742cd9b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:29:03.303835 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7b8fb0-74ef-43c1-88b5-507742cd9b4a None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:29:03.308729 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a300a08-7a94-4ebd-87b8-7fd8a25c7aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:29:03.309415 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a300a08-7a94-4ebd-87b8-7fd8a25c7aa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:03.313461 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a300a08-7a94-4ebd-87b8-7fd8a25c7aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:03.313461 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a300a08-7a94-4ebd-87b8-7fd8a25c7aa8 admin admin] PATH_INFO: `/v3/projects/4f3b24ad523f4bc0afa901454cfbb69a` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:03.313461 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a300a08-7a94-4ebd-87b8-7fd8a25c7aa8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f3b24ad523f4bc0afa901454cfbb69a)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:29:03.313736 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a300a08-7a94-4ebd-87b8-7fd8a25c7aa8 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:29:03.414683 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-6a300a08-7a94-4ebd-87b8-7fd8a25c7aa8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f3b24ad523f4bc0afa901454cfbb69a', 'request_id': 'req-6a300a08-7a94-4ebd-87b8-7fd8a25c7aa8'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:29:03.420982 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-6a300a08-7a94-4ebd-87b8-7fd8a25c7aa8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f3b24ad523f4bc0afa901454cfbb69a', 'request_id': 'req-6a300a08-7a94-4ebd-87b8-7fd8a25c7aa8'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:29:03.482438 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-6a300a08-7a94-4ebd-87b8-7fd8a25c7aa8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f3b24ad523f4bc0afa901454cfbb69a', 'request_id': 'req-6a300a08-7a94-4ebd-87b8-7fd8a25c7aa8'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:29:03.494180 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-6a300a08-7a94-4ebd-87b8-7fd8a25c7aa8 admin admin] The token cache is being invalidate because project 4f3b24ad523f4bc0afa901454cfbb69a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81747) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:29:03.494471 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-6a300a08-7a94-4ebd-87b8-7fd8a25c7aa8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:29:03.495104 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-6a300a08-7a94-4ebd-87b8-7fd8a25c7aa8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:29:03.496610 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1229/2449] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:29:03 2022] DELETE /identity/v3/projects/4f3b24ad523f4bc0afa901454cfbb69a => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:29:03.500642 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fc5c86-5ff5-45e2-aaeb-bece47f08325 service neutron] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:29:03.511581 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fc5c86-5ff5-45e2-aaeb-bece47f08325 service neutron] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:29:03.570233 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e00626a-d650-4eca-9fb8-24aa44472b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:29:03.570869 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e00626a-d650-4eca-9fb8-24aa44472b8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:03.571145 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e00626a-d650-4eca-9fb8-24aa44472b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:03.571441 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e00626a-d650-4eca-9fb8-24aa44472b8d admin admin] PATH_INFO: `/v3/users/1a1764046a324cac81150aecca87bc10` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:03.573554 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e00626a-d650-4eca-9fb8-24aa44472b8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a1764046a324cac81150aecca87bc10)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:29:03.580281 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e00626a-d650-4eca-9fb8-24aa44472b8d admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:29:03.776011 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-5e00626a-d650-4eca-9fb8-24aa44472b8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a1764046a324cac81150aecca87bc10'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:29:03.776655 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-5e00626a-d650-4eca-9fb8-24aa44472b8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a1764046a324cac81150aecca87bc10'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:29:03.789506 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-5e00626a-d650-4eca-9fb8-24aa44472b8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a1764046a324cac81150aecca87bc10'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:29:03.793104 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:29:03.859888 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-5e00626a-d650-4eca-9fb8-24aa44472b8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a1764046a324cac81150aecca87bc10'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:29:03.860554 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-5e00626a-d650-4eca-9fb8-24aa44472b8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a1764046a324cac81150aecca87bc10'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:29:03.873298 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1221/2450] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:29:03 2022] DELETE /identity/v3/users/1a1764046a324cac81150aecca87bc10 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:29:04.249411 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a300a08-7a94-4ebd-87b8-7fd8a25c7aa8 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:29:04.251450 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-6a300a08-7a94-4ebd-87b8-7fd8a25c7aa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:29:04.265333 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a300a08-7a94-4ebd-87b8-7fd8a25c7aa8 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:29:04.278421 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:29:04.323520 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f273bd0-a8e2-4180-b4c2-576f591901b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:29:04.324220 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f273bd0-a8e2-4180-b4c2-576f591901b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:04.324498 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f273bd0-a8e2-4180-b4c2-576f591901b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:04.324769 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f273bd0-a8e2-4180-b4c2-576f591901b1 admin admin] PATH_INFO: `/v3/projects/ced59da82b9945638f9f5ebb447eb30d` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:04.325906 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f273bd0-a8e2-4180-b4c2-576f591901b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ced59da82b9945638f9f5ebb447eb30d)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:29:04.328818 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f273bd0-a8e2-4180-b4c2-576f591901b1 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:29:04.468285 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-8f273bd0-a8e2-4180-b4c2-576f591901b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ced59da82b9945638f9f5ebb447eb30d', 'request_id': 'req-8f273bd0-a8e2-4180-b4c2-576f591901b1'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:29:04.474851 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-8f273bd0-a8e2-4180-b4c2-576f591901b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ced59da82b9945638f9f5ebb447eb30d', 'request_id': 'req-8f273bd0-a8e2-4180-b4c2-576f591901b1'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:29:04.545885 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-8f273bd0-a8e2-4180-b4c2-576f591901b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ced59da82b9945638f9f5ebb447eb30d', 'request_id': 'req-8f273bd0-a8e2-4180-b4c2-576f591901b1'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:29:04.557787 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-8f273bd0-a8e2-4180-b4c2-576f591901b1 admin admin] The token cache is being invalidate because project ced59da82b9945638f9f5ebb447eb30d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81747) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:29:04.558158 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-8f273bd0-a8e2-4180-b4c2-576f591901b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:29:04.558854 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-8f273bd0-a8e2-4180-b4c2-576f591901b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:29:04.560511 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1230/2451] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:29:04 2022] DELETE /identity/v3/projects/ced59da82b9945638f9f5ebb447eb30d => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:29:04.565961 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff17d64-94c3-4c88-ada8-0926f2cc30fe None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:04.566427 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff17d64-94c3-4c88-ada8-0926f2cc30fe None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:04.566581 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff17d64-94c3-4c88-ada8-0926f2cc30fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:04.585955 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-aff17d64-94c3-4c88-ada8-0926f2cc30fe None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:29:04.587150 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-aff17d64-94c3-4c88-ada8-0926f2cc30fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:29:04.645198 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1222/2452] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:29:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:29:04.653060 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3818c67c-cfb8-4ddb-8e60-be4944e879f4 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:04.653350 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3818c67c-cfb8-4ddb-8e60-be4944e879f4 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:04.653752 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3818c67c-cfb8-4ddb-8e60-be4944e879f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:04.675343 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-3818c67c-cfb8-4ddb-8e60-be4944e879f4 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:29:04.676567 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-3818c67c-cfb8-4ddb-8e60-be4944e879f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:29:04.684609 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1231/2453] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:29:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:29:04.689513 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff17d64-94c3-4c88-ada8-0926f2cc30fe None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:29:04.699882 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff17d64-94c3-4c88-ada8-0926f2cc30fe None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:29:04.704734 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e543f17f-c6ab-46ac-ba75-e8a38cb9b8fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:29:04.705517 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e543f17f-c6ab-46ac-ba75-e8a38cb9b8fe admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:04.705799 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e543f17f-c6ab-46ac-ba75-e8a38cb9b8fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:04.706073 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e543f17f-c6ab-46ac-ba75-e8a38cb9b8fe admin admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:04.706721 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e543f17f-c6ab-46ac-ba75-e8a38cb9b8fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:29:04.707026 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e543f17f-c6ab-46ac-ba75-e8a38cb9b8fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:29:04.708675 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e543f17f-c6ab-46ac-ba75-e8a38cb9b8fe admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:29:04.720353 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1223/2454] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Oct 12 00:29:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:29:04.725016 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3818c67c-cfb8-4ddb-8e60-be4944e879f4 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:29:04.734872 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3818c67c-cfb8-4ddb-8e60-be4944e879f4 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:29:04.739590 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d1d476-bed9-43c1-991e-60d1558395b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:29:04.740190 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d1d476-bed9-43c1-991e-60d1558395b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:04.740468 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d1d476-bed9-43c1-991e-60d1558395b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:04.740738 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d1d476-bed9-43c1-991e-60d1558395b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:04.741361 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d1d476-bed9-43c1-991e-60d1558395b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:29:04.743313 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d1d476-bed9-43c1-991e-60d1558395b4 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:29:04.838265 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1232/2455] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:29:04 2022] POST /identity/v3/projects => generated 375 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:29:04.842611 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e543f17f-c6ab-46ac-ba75-e8a38cb9b8fe admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:29:04.852552 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e543f17f-c6ab-46ac-ba75-e8a38cb9b8fe admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:29:04.915586 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9bcbb9-dcc6-4625-9393-e58d99debccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:29:04.916208 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9bcbb9-dcc6-4625-9393-e58d99debccc admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:04.916558 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9bcbb9-dcc6-4625-9393-e58d99debccc admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:04.916903 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9bcbb9-dcc6-4625-9393-e58d99debccc admin admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:04.917523 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9bcbb9-dcc6-4625-9393-e58d99debccc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:29:04.919643 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9bcbb9-dcc6-4625-9393-e58d99debccc admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:29:05.026002 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1224/2456] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:29:04 2022] POST /identity/v3/users => generated 288 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:29:05.029774 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d1d476-bed9-43c1-991e-60d1558395b4 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:29:05.041343 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d1d476-bed9-43c1-991e-60d1558395b4 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:29:05.046184 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eab50ea-85f3-4b21-a5d9-e7498570781b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:29:05.047464 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eab50ea-85f3-4b21-a5d9-e7498570781b admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:05.047884 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eab50ea-85f3-4b21-a5d9-e7498570781b admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:05.048184 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eab50ea-85f3-4b21-a5d9-e7498570781b admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:05.048849 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eab50ea-85f3-4b21-a5d9-e7498570781b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:29:05.050439 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eab50ea-85f3-4b21-a5d9-e7498570781b admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:29:05.061011 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1233/2457] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:29:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:29:05.065147 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9bcbb9-dcc6-4625-9393-e58d99debccc admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:29:05.075060 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9bcbb9-dcc6-4625-9393-e58d99debccc admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:29:05.079670 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eba3939-4b68-4b0c-9f20-b6155c68e7b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:29:05.080266 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba3939-4b68-4b0c-9f20-b6155c68e7b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:05.080542 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba3939-4b68-4b0c-9f20-b6155c68e7b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:05.084846 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba3939-4b68-4b0c-9f20-b6155c68e7b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:05.084846 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eba3939-4b68-4b0c-9f20-b6155c68e7b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:29:05.084846 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eba3939-4b68-4b0c-9f20-b6155c68e7b3 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:29:05.092745 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1225/2458] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:29:05 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:29:05.097524 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eab50ea-85f3-4b21-a5d9-e7498570781b admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:29:05.107872 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eab50ea-85f3-4b21-a5d9-e7498570781b admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:29:05.113197 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2afdc10-23b5-41c3-86cf-df003d6ff9ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:29:05.113850 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2afdc10-23b5-41c3-86cf-df003d6ff9ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:05.114118 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2afdc10-23b5-41c3-86cf-df003d6ff9ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:05.114417 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2afdc10-23b5-41c3-86cf-df003d6ff9ac admin admin] PATH_INFO: `/v3/projects/3661fb9353db4a539ea34dd6cd6aa383/users/14377f3dc471429299a282e33be63380/roles/ef689f96442945e8a7bc8cbfee13d909` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:05.129829 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2afdc10-23b5-41c3-86cf-df003d6ff9ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=3661fb9353db4a539ea34dd6cd6aa383, user_id=14377f3dc471429299a282e33be63380, role_id=ef689f96442945e8a7bc8cbfee13d909)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:29:05.132732 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2afdc10-23b5-41c3-86cf-df003d6ff9ac admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:29:05.220107 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1234/2459] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:29:05 2022] PUT /identity/v3/projects/3661fb9353db4a539ea34dd6cd6aa383/users/14377f3dc471429299a282e33be63380/roles/ef689f96442945e8a7bc8cbfee13d909 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 00:29:05.232299 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530d8f61-c910-4bc0-b803-3d0d689047ee None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:05.232582 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530d8f61-c910-4bc0-b803-3d0d689047ee None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:05.232857 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530d8f61-c910-4bc0-b803-3d0d689047ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:05.260971 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-530d8f61-c910-4bc0-b803-3d0d689047ee None None] MFA Rules not processed for user `14377f3dc471429299a282e33be63380`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:29:05.262194 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-530d8f61-c910-4bc0-b803-3d0d689047ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:29:05.303336 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:29:05.326468 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1226/2460] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:29:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:29:05.331530 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2afdc10-23b5-41c3-86cf-df003d6ff9ac admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:29:05.376738 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:29:05.394142 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2afdc10-23b5-41c3-86cf-df003d6ff9ac admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:29:05.398865 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9fdd3e-9df4-4fb8-b7ce-f3fba75864de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:29:05.399494 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9fdd3e-9df4-4fb8-b7ce-f3fba75864de admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:05.399771 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9fdd3e-9df4-4fb8-b7ce-f3fba75864de admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:05.400066 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9fdd3e-9df4-4fb8-b7ce-f3fba75864de admin admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:05.400690 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9fdd3e-9df4-4fb8-b7ce-f3fba75864de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:29:05.402836 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9fdd3e-9df4-4fb8-b7ce-f3fba75864de admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:29:05.489750 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1235/2461] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:29:05 2022] POST /identity/v3/projects => generated 373 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:29:05.493841 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530d8f61-c910-4bc0-b803-3d0d689047ee None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:29:05.504435 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530d8f61-c910-4bc0-b803-3d0d689047ee None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:29:05.562730 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66630c1-fa9b-4ccf-aee0-345d833d70c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:29:05.563312 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66630c1-fa9b-4ccf-aee0-345d833d70c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:05.563744 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66630c1-fa9b-4ccf-aee0-345d833d70c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:05.564017 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66630c1-fa9b-4ccf-aee0-345d833d70c9 admin admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:05.564636 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66630c1-fa9b-4ccf-aee0-345d833d70c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:29:05.566625 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66630c1-fa9b-4ccf-aee0-345d833d70c9 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:29:05.682540 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1227/2462] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:29:05 2022] POST /identity/v3/users => generated 293 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:29:05.686815 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9fdd3e-9df4-4fb8-b7ce-f3fba75864de admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:29:05.697447 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9fdd3e-9df4-4fb8-b7ce-f3fba75864de admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:29:05.702141 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e271706-157a-44a7-a574-7d765faf5b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:29:05.702735 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e271706-157a-44a7-a574-7d765faf5b73 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:05.703614 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e271706-157a-44a7-a574-7d765faf5b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:05.703921 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e271706-157a-44a7-a574-7d765faf5b73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:05.704751 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e271706-157a-44a7-a574-7d765faf5b73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:29:05.706348 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e271706-157a-44a7-a574-7d765faf5b73 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:29:05.716757 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1236/2463] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:29:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:29:05.720765 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66630c1-fa9b-4ccf-aee0-345d833d70c9 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:29:05.730760 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66630c1-fa9b-4ccf-aee0-345d833d70c9 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:29:05.735480 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db4f7ae-4a54-4686-aad0-0a618b204be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:29:05.736134 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db4f7ae-4a54-4686-aad0-0a618b204be7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:05.736408 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db4f7ae-4a54-4686-aad0-0a618b204be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:05.736679 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db4f7ae-4a54-4686-aad0-0a618b204be7 admin admin] PATH_INFO: `/v3/projects/0a3917c90eab4c5999ddce8417c33e47/users/07a9c8de0bea449e870304debe41d072/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:05.752645 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db4f7ae-4a54-4686-aad0-0a618b204be7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a3917c90eab4c5999ddce8417c33e47, user_id=07a9c8de0bea449e870304debe41d072, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:29:05.755355 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db4f7ae-4a54-4686-aad0-0a618b204be7 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:29:05.865176 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1228/2464] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:29:05 2022] PUT /identity/v3/projects/0a3917c90eab4c5999ddce8417c33e47/users/07a9c8de0bea449e870304debe41d072/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:29:05.869137 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e271706-157a-44a7-a574-7d765faf5b73 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:29:05.879848 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e271706-157a-44a7-a574-7d765faf5b73 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:29:05.935136 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b8989a-b47b-4afb-9def-e973182fdf2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:29:05.935773 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b8989a-b47b-4afb-9def-e973182fdf2f admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:05.936050 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b8989a-b47b-4afb-9def-e973182fdf2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:05.936319 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b8989a-b47b-4afb-9def-e973182fdf2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:05.936945 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b8989a-b47b-4afb-9def-e973182fdf2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:29:05.939024 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b8989a-b47b-4afb-9def-e973182fdf2f admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:29:05.948531 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1237/2465] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:29:05 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:29:05.952757 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db4f7ae-4a54-4686-aad0-0a618b204be7 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:29:05.963537 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db4f7ae-4a54-4686-aad0-0a618b204be7 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:29:05.968202 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614bc6cd-2f40-4db0-b58d-72b8c03d4cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:29:05.968926 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614bc6cd-2f40-4db0-b58d-72b8c03d4cbd admin admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:05.969226 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614bc6cd-2f40-4db0-b58d-72b8c03d4cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:05.969496 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614bc6cd-2f40-4db0-b58d-72b8c03d4cbd admin admin] PATH_INFO: `/v3/projects/0a3917c90eab4c5999ddce8417c33e47/users/07a9c8de0bea449e870304debe41d072/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:05.971572 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614bc6cd-2f40-4db0-b58d-72b8c03d4cbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a3917c90eab4c5999ddce8417c33e47, user_id=07a9c8de0bea449e870304debe41d072, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:29:05.974390 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614bc6cd-2f40-4db0-b58d-72b8c03d4cbd admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:29:05.984744 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1229/2466] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:29:05 2022] PUT /identity/v3/projects/0a3917c90eab4c5999ddce8417c33e47/users/07a9c8de0bea449e870304debe41d072/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:29:05.994714 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f985e16-f801-4af8-b8fc-435ce4bcf688 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:05.994987 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f985e16-f801-4af8-b8fc-435ce4bcf688 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:05.995256 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f985e16-f801-4af8-b8fc-435ce4bcf688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:06.023110 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-5f985e16-f801-4af8-b8fc-435ce4bcf688 None None] MFA Rules not processed for user `07a9c8de0bea449e870304debe41d072`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:29:06.024357 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-5f985e16-f801-4af8-b8fc-435ce4bcf688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:29:06.089844 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1238/2467] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:29:05 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:29:06.100703 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614bc6cd-2f40-4db0-b58d-72b8c03d4cbd admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:29:06.102346 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-614bc6cd-2f40-4db0-b58d-72b8c03d4cbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:29:06.158250 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614bc6cd-2f40-4db0-b58d-72b8c03d4cbd admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:29:06.161937 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03802d4b-35f2-4a12-a692-34be443bc602 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:29:06.162502 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03802d4b-35f2-4a12-a692-34be443bc602 service cinder] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:06.162774 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03802d4b-35f2-4a12-a692-34be443bc602 service cinder] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:06.163042 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03802d4b-35f2-4a12-a692-34be443bc602 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:06.184995 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f985e16-f801-4af8-b8fc-435ce4bcf688 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:29:06.186379 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-5f985e16-f801-4af8-b8fc-435ce4bcf688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:29:06.212055 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:29:06.221216 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03802d4b-35f2-4a12-a692-34be443bc602 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:29:06.222659 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03802d4b-35f2-4a12-a692-34be443bc602 service cinder] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:29:06.227868 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:29:06.231807 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1230/2468] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:29:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:29:06.252440 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f985e16-f801-4af8-b8fc-435ce4bcf688 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:29:06.257108 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a845b69c-5ee2-4cf1-9338-0f0f50255d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:29:06.257746 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a845b69c-5ee2-4cf1-9338-0f0f50255d21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:06.258017 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a845b69c-5ee2-4cf1-9338-0f0f50255d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:06.258287 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a845b69c-5ee2-4cf1-9338-0f0f50255d21 admin admin] PATH_INFO: `/v3/users/0164e18368ca40828ce322f396bc5999` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:06.259481 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a845b69c-5ee2-4cf1-9338-0f0f50255d21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0164e18368ca40828ce322f396bc5999)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:29:06.261383 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a845b69c-5ee2-4cf1-9338-0f0f50255d21 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:29:06.421543 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-a845b69c-5ee2-4cf1-9338-0f0f50255d21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0164e18368ca40828ce322f396bc5999'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:29:06.422774 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-a845b69c-5ee2-4cf1-9338-0f0f50255d21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0164e18368ca40828ce322f396bc5999'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:29:06.434573 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-a845b69c-5ee2-4cf1-9338-0f0f50255d21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0164e18368ca40828ce322f396bc5999'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:29:06.510293 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-a845b69c-5ee2-4cf1-9338-0f0f50255d21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0164e18368ca40828ce322f396bc5999'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:29:06.510947 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-a845b69c-5ee2-4cf1-9338-0f0f50255d21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0164e18368ca40828ce322f396bc5999'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:29:06.523845 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1239/2469] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:29:06 2022] DELETE /identity/v3/users/0164e18368ca40828ce322f396bc5999 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 00:29:06.528493 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7502b8-8157-4436-b464-eae03c5d4fd4 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:06.528797 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7502b8-8157-4436-b464-eae03c5d4fd4 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:06.529069 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7502b8-8157-4436-b464-eae03c5d4fd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:06.550597 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-6d7502b8-8157-4436-b464-eae03c5d4fd4 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:29:06.551848 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-6d7502b8-8157-4436-b464-eae03c5d4fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:29:06.609879 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1231/2470] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:29:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:29:06.619565 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a845b69c-5ee2-4cf1-9338-0f0f50255d21 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:29:06.620989 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-a845b69c-5ee2-4cf1-9338-0f0f50255d21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:29:06.679447 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a845b69c-5ee2-4cf1-9338-0f0f50255d21 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:29:06.683037 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e4ed2f-b6cb-46c8-9fa5-d39f53a5e1fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:29:06.683667 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e4ed2f-b6cb-46c8-9fa5-d39f53a5e1fb service neutron] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:06.683947 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e4ed2f-b6cb-46c8-9fa5-d39f53a5e1fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:06.684216 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e4ed2f-b6cb-46c8-9fa5-d39f53a5e1fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:06.693198 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e4ed2f-b6cb-46c8-9fa5-d39f53a5e1fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:29:06.694617 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e4ed2f-b6cb-46c8-9fa5-d39f53a5e1fb service neutron] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:29:06.704049 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1240/2471] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:29:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:29:07.089441 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7502b8-8157-4436-b464-eae03c5d4fd4 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:29:07.091103 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-6d7502b8-8157-4436-b464-eae03c5d4fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:29:07.104970 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7502b8-8157-4436-b464-eae03c5d4fd4 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:29:07.110168 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c50df39-4997-4d4d-aa0b-0575641da236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:29:07.111027 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c50df39-4997-4d4d-aa0b-0575641da236 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:07.111309 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c50df39-4997-4d4d-aa0b-0575641da236 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:07.111769 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c50df39-4997-4d4d-aa0b-0575641da236 admin admin] PATH_INFO: `/v3/projects/2366d2cda6374336ad68b9e67220f620` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:07.112907 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c50df39-4997-4d4d-aa0b-0575641da236 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2366d2cda6374336ad68b9e67220f620)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:29:07.114872 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c50df39-4997-4d4d-aa0b-0575641da236 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:29:07.219249 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-2c50df39-4997-4d4d-aa0b-0575641da236 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2366d2cda6374336ad68b9e67220f620', 'request_id': 'req-2c50df39-4997-4d4d-aa0b-0575641da236'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:29:07.226017 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-2c50df39-4997-4d4d-aa0b-0575641da236 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2366d2cda6374336ad68b9e67220f620', 'request_id': 'req-2c50df39-4997-4d4d-aa0b-0575641da236'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:29:07.293617 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-2c50df39-4997-4d4d-aa0b-0575641da236 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2366d2cda6374336ad68b9e67220f620', 'request_id': 'req-2c50df39-4997-4d4d-aa0b-0575641da236'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:29:07.304334 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-2c50df39-4997-4d4d-aa0b-0575641da236 admin admin] The token cache is being invalidate because project 2366d2cda6374336ad68b9e67220f620 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81746) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:29:07.304645 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-2c50df39-4997-4d4d-aa0b-0575641da236 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:29:07.305231 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-2c50df39-4997-4d4d-aa0b-0575641da236 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:29:07.306716 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1232/2472] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:29:07 2022] DELETE /identity/v3/projects/2366d2cda6374336ad68b9e67220f620 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:29:07.311578 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8859c224-ccb3-4f71-ab6e-dccbffbae046 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:07.311859 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8859c224-ccb3-4f71-ab6e-dccbffbae046 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:07.312137 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8859c224-ccb3-4f71-ab6e-dccbffbae046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:07.332156 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-8859c224-ccb3-4f71-ab6e-dccbffbae046 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:29:07.333338 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-8859c224-ccb3-4f71-ab6e-dccbffbae046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:29:07.383622 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:29:07.392395 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1241/2473] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:29:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:29:07.400440 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e80dd-48a5-4fab-9dd3-e0e559f68664 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:07.400730 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e80dd-48a5-4fab-9dd3-e0e559f68664 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:07.401000 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e80dd-48a5-4fab-9dd3-e0e559f68664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:07.420396 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-cf4e80dd-48a5-4fab-9dd3-e0e559f68664 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:29:07.421595 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-cf4e80dd-48a5-4fab-9dd3-e0e559f68664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:29:07.429451 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1233/2474] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:29:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:29:07.433789 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8859c224-ccb3-4f71-ab6e-dccbffbae046 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:29:07.443791 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8859c224-ccb3-4f71-ab6e-dccbffbae046 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:29:07.448456 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d1e64c-699c-4728-a2ea-78750945b737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:29:07.449034 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d1e64c-699c-4728-a2ea-78750945b737 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:07.449308 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d1e64c-699c-4728-a2ea-78750945b737 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:07.449576 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d1e64c-699c-4728-a2ea-78750945b737 admin admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:07.450213 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d1e64c-699c-4728-a2ea-78750945b737 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:29:07.450510 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d1e64c-699c-4728-a2ea-78750945b737 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:29:07.452140 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d1e64c-699c-4728-a2ea-78750945b737 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:29:07.463989 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1242/2475] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Oct 12 00:29:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:29:07.468220 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4e80dd-48a5-4fab-9dd3-e0e559f68664 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:29:07.477559 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4e80dd-48a5-4fab-9dd3-e0e559f68664 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:29:07.482267 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d6f7db-12d5-461a-ac08-797e0b43df01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:29:07.482862 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d6f7db-12d5-461a-ac08-797e0b43df01 admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:07.483138 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d6f7db-12d5-461a-ac08-797e0b43df01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:07.483435 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d6f7db-12d5-461a-ac08-797e0b43df01 admin admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:07.484058 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d6f7db-12d5-461a-ac08-797e0b43df01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:29:07.486429 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d6f7db-12d5-461a-ac08-797e0b43df01 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:29:07.575773 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1234/2476] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:29:07 2022] POST /identity/v3/projects => generated 377 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:29:07.580144 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d1e64c-699c-4728-a2ea-78750945b737 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:29:07.590210 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d1e64c-699c-4728-a2ea-78750945b737 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:29:07.648553 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0349b6-89fc-401e-9fad-7ab8872194dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:29:07.649153 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0349b6-89fc-401e-9fad-7ab8872194dd admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:07.649424 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0349b6-89fc-401e-9fad-7ab8872194dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:07.649689 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0349b6-89fc-401e-9fad-7ab8872194dd admin admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:07.650317 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0349b6-89fc-401e-9fad-7ab8872194dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:29:07.652263 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0349b6-89fc-401e-9fad-7ab8872194dd admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:29:07.717149 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1243/2477] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:29:07 2022] POST /identity/v3/users => generated 289 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:29:07.721269 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d6f7db-12d5-461a-ac08-797e0b43df01 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:29:07.731566 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d6f7db-12d5-461a-ac08-797e0b43df01 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:29:07.736772 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff8f1f1-d6a7-4dbc-959a-f24af3ed84dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:29:07.737369 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff8f1f1-d6a7-4dbc-959a-f24af3ed84dd admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:07.737645 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff8f1f1-d6a7-4dbc-959a-f24af3ed84dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:07.737913 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff8f1f1-d6a7-4dbc-959a-f24af3ed84dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:07.738548 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff8f1f1-d6a7-4dbc-959a-f24af3ed84dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:29:07.740161 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff8f1f1-d6a7-4dbc-959a-f24af3ed84dd admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:29:07.750766 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1235/2478] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:29:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:29:07.754652 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0349b6-89fc-401e-9fad-7ab8872194dd admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:29:07.764603 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0349b6-89fc-401e-9fad-7ab8872194dd admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:29:07.769411 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9bdd1c-6a5d-4f6b-8265-d7c83e2cd3ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:29:07.770020 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9bdd1c-6a5d-4f6b-8265-d7c83e2cd3ac admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:07.770299 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9bdd1c-6a5d-4f6b-8265-d7c83e2cd3ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:07.770566 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9bdd1c-6a5d-4f6b-8265-d7c83e2cd3ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:07.771275 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9bdd1c-6a5d-4f6b-8265-d7c83e2cd3ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:29:07.772906 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9bdd1c-6a5d-4f6b-8265-d7c83e2cd3ac admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:29:07.783260 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1244/2479] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:29:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 00:29:07.787188 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff8f1f1-d6a7-4dbc-959a-f24af3ed84dd admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:29:07.798212 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff8f1f1-d6a7-4dbc-959a-f24af3ed84dd admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:29:07.803653 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2960cf0a-da0e-4ca5-96c9-157a7c1473c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:29:07.803653 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2960cf0a-da0e-4ca5-96c9-157a7c1473c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:07.803873 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2960cf0a-da0e-4ca5-96c9-157a7c1473c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:07.804116 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2960cf0a-da0e-4ca5-96c9-157a7c1473c4 admin admin] PATH_INFO: `/v3/projects/ec802857e5dd4a5cbe5b57fed385196a/users/257b2e9ebdc14c77995a42164072b298/roles/ef689f96442945e8a7bc8cbfee13d909` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:07.819283 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2960cf0a-da0e-4ca5-96c9-157a7c1473c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec802857e5dd4a5cbe5b57fed385196a, user_id=257b2e9ebdc14c77995a42164072b298, role_id=ef689f96442945e8a7bc8cbfee13d909)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:29:07.822087 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2960cf0a-da0e-4ca5-96c9-157a7c1473c4 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:29:07.894768 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1236/2480] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:29:07 2022] PUT /identity/v3/projects/ec802857e5dd4a5cbe5b57fed385196a/users/257b2e9ebdc14c77995a42164072b298/roles/ef689f96442945e8a7bc8cbfee13d909 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:29:07.906757 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5d9316-8b32-4280-aaa3-cb80761aa4a6 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:07.907285 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5d9316-8b32-4280-aaa3-cb80761aa4a6 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:07.907585 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5d9316-8b32-4280-aaa3-cb80761aa4a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:07.938346 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-3b5d9316-8b32-4280-aaa3-cb80761aa4a6 None None] MFA Rules not processed for user `257b2e9ebdc14c77995a42164072b298`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:29:07.939653 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-3b5d9316-8b32-4280-aaa3-cb80761aa4a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:29:08.015019 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1245/2481] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:29:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:29:08.027145 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2960cf0a-da0e-4ca5-96c9-157a7c1473c4 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:29:08.028932 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-2960cf0a-da0e-4ca5-96c9-157a7c1473c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:29:08.040701 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:29:08.085378 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2960cf0a-da0e-4ca5-96c9-157a7c1473c4 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:29:08.089270 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524470f8-d4ee-41c2-98e7-8d0a05e773d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:29:08.089876 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524470f8-d4ee-41c2-98e7-8d0a05e773d9 service cinder] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:08.090157 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524470f8-d4ee-41c2-98e7-8d0a05e773d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:08.090426 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524470f8-d4ee-41c2-98e7-8d0a05e773d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:08.100998 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524470f8-d4ee-41c2-98e7-8d0a05e773d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:29:08.102456 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524470f8-d4ee-41c2-98e7-8d0a05e773d9 service cinder] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:29:08.112663 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1237/2482] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:29:08 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:29:09.285906 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5d9316-8b32-4280-aaa3-cb80761aa4a6 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:29:09.287432 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-3b5d9316-8b32-4280-aaa3-cb80761aa4a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:29:09.346468 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:29:09.351867 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5d9316-8b32-4280-aaa3-cb80761aa4a6 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:29:09.358352 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3a7a72-758f-4b0b-8f70-02374e4ec347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:29:09.358352 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3a7a72-758f-4b0b-8f70-02374e4ec347 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:09.358352 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3a7a72-758f-4b0b-8f70-02374e4ec347 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:09.358352 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3a7a72-758f-4b0b-8f70-02374e4ec347 admin admin] PATH_INFO: `/v3/users/14377f3dc471429299a282e33be63380` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:09.358738 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3a7a72-758f-4b0b-8f70-02374e4ec347 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14377f3dc471429299a282e33be63380)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:29:09.361209 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3a7a72-758f-4b0b-8f70-02374e4ec347 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:29:09.646070 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-ab3a7a72-758f-4b0b-8f70-02374e4ec347 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14377f3dc471429299a282e33be63380'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:29:09.646839 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-ab3a7a72-758f-4b0b-8f70-02374e4ec347 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14377f3dc471429299a282e33be63380'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:29:09.659585 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-ab3a7a72-758f-4b0b-8f70-02374e4ec347 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14377f3dc471429299a282e33be63380'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:29:09.716357 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-ab3a7a72-758f-4b0b-8f70-02374e4ec347 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14377f3dc471429299a282e33be63380'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:29:09.716970 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-ab3a7a72-758f-4b0b-8f70-02374e4ec347 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14377f3dc471429299a282e33be63380'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:29:09.729975 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1246/2483] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:29:09 2022] DELETE /identity/v3/users/14377f3dc471429299a282e33be63380 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:29:09.734540 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dd0b26-bec2-49df-aef2-cf833e074798 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:09.735668 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dd0b26-bec2-49df-aef2-cf833e074798 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:09.735668 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dd0b26-bec2-49df-aef2-cf833e074798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:09.754858 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-61dd0b26-bec2-49df-aef2-cf833e074798 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:29:09.756147 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-61dd0b26-bec2-49df-aef2-cf833e074798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:29:09.815916 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1238/2484] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:29:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:29:09.829124 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3a7a72-758f-4b0b-8f70-02374e4ec347 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:29:09.830588 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-ab3a7a72-758f-4b0b-8f70-02374e4ec347 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:29:09.889335 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3a7a72-758f-4b0b-8f70-02374e4ec347 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:29:09.893496 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a245f873-9fe8-4222-bc35-974619794b2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:29:09.899713 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a245f873-9fe8-4222-bc35-974619794b2f service neutron] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:09.899713 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a245f873-9fe8-4222-bc35-974619794b2f service neutron] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:09.899713 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a245f873-9fe8-4222-bc35-974619794b2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:09.904182 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a245f873-9fe8-4222-bc35-974619794b2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:29:09.905601 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a245f873-9fe8-4222-bc35-974619794b2f service neutron] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:29:09.916207 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1247/2485] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:29:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 00:29:10.295664 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dd0b26-bec2-49df-aef2-cf833e074798 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:29:10.297350 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-61dd0b26-bec2-49df-aef2-cf833e074798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:29:10.311103 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dd0b26-bec2-49df-aef2-cf833e074798 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:29:10.316438 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca7b48d-4c4b-4766-8210-d8ca9d7617df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:29:10.317167 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca7b48d-4c4b-4766-8210-d8ca9d7617df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:10.317644 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca7b48d-4c4b-4766-8210-d8ca9d7617df admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:10.317997 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca7b48d-4c4b-4766-8210-d8ca9d7617df admin admin] PATH_INFO: `/v3/projects/3661fb9353db4a539ea34dd6cd6aa383` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:10.319339 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca7b48d-4c4b-4766-8210-d8ca9d7617df admin admin] RBAC: Authorizing `identity:delete_project(project_id=3661fb9353db4a539ea34dd6cd6aa383)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:29:10.321529 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca7b48d-4c4b-4766-8210-d8ca9d7617df admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:29:10.430591 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-7ca7b48d-4c4b-4766-8210-d8ca9d7617df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3661fb9353db4a539ea34dd6cd6aa383', 'request_id': 'req-7ca7b48d-4c4b-4766-8210-d8ca9d7617df'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:29:10.436692 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-7ca7b48d-4c4b-4766-8210-d8ca9d7617df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3661fb9353db4a539ea34dd6cd6aa383', 'request_id': 'req-7ca7b48d-4c4b-4766-8210-d8ca9d7617df'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:29:10.516950 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-7ca7b48d-4c4b-4766-8210-d8ca9d7617df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3661fb9353db4a539ea34dd6cd6aa383', 'request_id': 'req-7ca7b48d-4c4b-4766-8210-d8ca9d7617df'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:29:10.527545 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-7ca7b48d-4c4b-4766-8210-d8ca9d7617df admin admin] The token cache is being invalidate because project 3661fb9353db4a539ea34dd6cd6aa383 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81746) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:29:10.527828 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-7ca7b48d-4c4b-4766-8210-d8ca9d7617df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:29:10.528485 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-7ca7b48d-4c4b-4766-8210-d8ca9d7617df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:29:10.529950 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1239/2486] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:29:10 2022] DELETE /identity/v3/projects/3661fb9353db4a539ea34dd6cd6aa383 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:29:10.533919 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a245f873-9fe8-4222-bc35-974619794b2f service neutron] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:29:10.543752 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a245f873-9fe8-4222-bc35-974619794b2f service neutron] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:29:10.600176 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e38bdd5-dfbb-4b89-a678-655911926137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:29:10.600799 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e38bdd5-dfbb-4b89-a678-655911926137 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:10.601074 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e38bdd5-dfbb-4b89-a678-655911926137 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:10.601343 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e38bdd5-dfbb-4b89-a678-655911926137 admin admin] PATH_INFO: `/v3/users/07a9c8de0bea449e870304debe41d072` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:10.602448 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e38bdd5-dfbb-4b89-a678-655911926137 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07a9c8de0bea449e870304debe41d072)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:29:10.604369 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e38bdd5-dfbb-4b89-a678-655911926137 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:29:10.626090 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:29:10.783316 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-1e38bdd5-dfbb-4b89-a678-655911926137 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07a9c8de0bea449e870304debe41d072'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:29:10.783930 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-1e38bdd5-dfbb-4b89-a678-655911926137 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07a9c8de0bea449e870304debe41d072'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:29:10.796254 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-1e38bdd5-dfbb-4b89-a678-655911926137 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07a9c8de0bea449e870304debe41d072'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:29:10.871082 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-1e38bdd5-dfbb-4b89-a678-655911926137 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07a9c8de0bea449e870304debe41d072'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:29:10.871713 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-1e38bdd5-dfbb-4b89-a678-655911926137 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07a9c8de0bea449e870304debe41d072'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:29:10.884347 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1248/2487] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:29:10 2022] DELETE /identity/v3/users/07a9c8de0bea449e870304debe41d072 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:29:11.286936 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca7b48d-4c4b-4766-8210-d8ca9d7617df admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:29:11.286936 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-7ca7b48d-4c4b-4766-8210-d8ca9d7617df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:29:11.299175 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca7b48d-4c4b-4766-8210-d8ca9d7617df admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:29:11.332266 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:29:11.356912 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274d0935-a010-4f40-9795-0504bbaaf0ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:29:11.357545 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d0935-a010-4f40-9795-0504bbaaf0ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:11.357821 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d0935-a010-4f40-9795-0504bbaaf0ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:11.358089 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d0935-a010-4f40-9795-0504bbaaf0ae admin admin] PATH_INFO: `/v3/projects/0a3917c90eab4c5999ddce8417c33e47` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:11.359223 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274d0935-a010-4f40-9795-0504bbaaf0ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a3917c90eab4c5999ddce8417c33e47)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:29:11.361227 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274d0935-a010-4f40-9795-0504bbaaf0ae admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:29:11.470466 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-274d0935-a010-4f40-9795-0504bbaaf0ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a3917c90eab4c5999ddce8417c33e47', 'request_id': 'req-274d0935-a010-4f40-9795-0504bbaaf0ae'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:29:11.477433 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-274d0935-a010-4f40-9795-0504bbaaf0ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a3917c90eab4c5999ddce8417c33e47', 'request_id': 'req-274d0935-a010-4f40-9795-0504bbaaf0ae'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:29:11.562215 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-274d0935-a010-4f40-9795-0504bbaaf0ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a3917c90eab4c5999ddce8417c33e47', 'request_id': 'req-274d0935-a010-4f40-9795-0504bbaaf0ae'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:29:11.573791 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-274d0935-a010-4f40-9795-0504bbaaf0ae admin admin] The token cache is being invalidate because project 0a3917c90eab4c5999ddce8417c33e47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81746) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:29:11.574079 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-274d0935-a010-4f40-9795-0504bbaaf0ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:29:11.574694 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-274d0935-a010-4f40-9795-0504bbaaf0ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:29:11.576226 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1240/2488] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:29:11 2022] DELETE /identity/v3/projects/0a3917c90eab4c5999ddce8417c33e47 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:29:11.581762 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c32b3b4-0d1a-45ee-b89b-1547ec54fd41 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:11.582237 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c32b3b4-0d1a-45ee-b89b-1547ec54fd41 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:11.582344 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c32b3b4-0d1a-45ee-b89b-1547ec54fd41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:11.602032 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-2c32b3b4-0d1a-45ee-b89b-1547ec54fd41 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:29:11.603215 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-2c32b3b4-0d1a-45ee-b89b-1547ec54fd41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:29:11.662630 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1249/2489] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:29:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:29:11.670667 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c17548-804f-432f-9138-d2169fe1a55f None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:11.670958 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c17548-804f-432f-9138-d2169fe1a55f None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:11.671238 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c17548-804f-432f-9138-d2169fe1a55f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:11.691692 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-03c17548-804f-432f-9138-d2169fe1a55f None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:29:11.692903 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-03c17548-804f-432f-9138-d2169fe1a55f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:29:11.700887 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1241/2490] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:29:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:29:11.705644 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c32b3b4-0d1a-45ee-b89b-1547ec54fd41 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:29:11.715345 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c32b3b4-0d1a-45ee-b89b-1547ec54fd41 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:29:11.719915 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08935977-de31-4aef-9cba-aef3ccfe2499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:29:11.720511 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08935977-de31-4aef-9cba-aef3ccfe2499 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:11.720785 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08935977-de31-4aef-9cba-aef3ccfe2499 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:11.721056 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08935977-de31-4aef-9cba-aef3ccfe2499 admin admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:11.721693 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08935977-de31-4aef-9cba-aef3ccfe2499 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:29:11.722022 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08935977-de31-4aef-9cba-aef3ccfe2499 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:29:11.723676 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08935977-de31-4aef-9cba-aef3ccfe2499 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:29:11.735312 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1250/2491] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Oct 12 00:29:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:29:11.739203 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c17548-804f-432f-9138-d2169fe1a55f None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:29:11.748634 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c17548-804f-432f-9138-d2169fe1a55f None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:29:11.753842 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c54975-4e41-47e4-ac2e-7ab54cb6afe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:29:11.754440 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c54975-4e41-47e4-ac2e-7ab54cb6afe9 admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:11.754718 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c54975-4e41-47e4-ac2e-7ab54cb6afe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:11.755059 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c54975-4e41-47e4-ac2e-7ab54cb6afe9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:11.755738 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c54975-4e41-47e4-ac2e-7ab54cb6afe9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:29:11.757694 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c54975-4e41-47e4-ac2e-7ab54cb6afe9 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:29:11.832998 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1242/2492] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:29:11 2022] POST /identity/v3/projects => generated 371 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:29:11.837348 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08935977-de31-4aef-9cba-aef3ccfe2499 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:29:11.847415 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08935977-de31-4aef-9cba-aef3ccfe2499 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:29:11.905195 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fcd83c-05ee-45be-8a98-64bb393d4f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:29:11.905808 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fcd83c-05ee-45be-8a98-64bb393d4f0d admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:11.906111 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fcd83c-05ee-45be-8a98-64bb393d4f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:11.906385 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fcd83c-05ee-45be-8a98-64bb393d4f0d admin admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:11.907004 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fcd83c-05ee-45be-8a98-64bb393d4f0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:29:11.908953 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fcd83c-05ee-45be-8a98-64bb393d4f0d admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:29:12.025126 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1251/2493] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:29:11 2022] POST /identity/v3/users => generated 286 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:29:12.029187 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c54975-4e41-47e4-ac2e-7ab54cb6afe9 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:29:12.039556 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c54975-4e41-47e4-ac2e-7ab54cb6afe9 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:29:12.044207 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512dd408-8953-42e1-9536-1e0c536e4128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:29:12.044802 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512dd408-8953-42e1-9536-1e0c536e4128 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:12.045102 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512dd408-8953-42e1-9536-1e0c536e4128 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:12.045398 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512dd408-8953-42e1-9536-1e0c536e4128 admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:12.046032 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512dd408-8953-42e1-9536-1e0c536e4128 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:29:12.047741 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512dd408-8953-42e1-9536-1e0c536e4128 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:29:12.058068 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1243/2494] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:29:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:29:12.061976 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fcd83c-05ee-45be-8a98-64bb393d4f0d admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:29:12.072824 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fcd83c-05ee-45be-8a98-64bb393d4f0d admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:29:12.077447 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7a1b0c-fc9f-4e68-8c75-b2d968b3b60e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:29:12.078036 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7a1b0c-fc9f-4e68-8c75-b2d968b3b60e admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:12.078308 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7a1b0c-fc9f-4e68-8c75-b2d968b3b60e admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:12.078575 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7a1b0c-fc9f-4e68-8c75-b2d968b3b60e admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:12.079202 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7a1b0c-fc9f-4e68-8c75-b2d968b3b60e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:29:12.080872 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7a1b0c-fc9f-4e68-8c75-b2d968b3b60e admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:29:12.091570 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1252/2495] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:29:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:29:12.095290 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512dd408-8953-42e1-9536-1e0c536e4128 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:29:12.105070 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512dd408-8953-42e1-9536-1e0c536e4128 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:29:12.109581 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0b6d8b-29dc-4498-9e6e-70573948d73b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:29:12.110225 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0b6d8b-29dc-4498-9e6e-70573948d73b admin admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:12.110504 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0b6d8b-29dc-4498-9e6e-70573948d73b admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:12.110778 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0b6d8b-29dc-4498-9e6e-70573948d73b admin admin] PATH_INFO: `/v3/projects/cdece90b26c74eb89113d317b1609e92/users/c1ed99c8700a49bbb9ef9d4a13515c17/roles/ef689f96442945e8a7bc8cbfee13d909` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:12.125762 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0b6d8b-29dc-4498-9e6e-70573948d73b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdece90b26c74eb89113d317b1609e92, user_id=c1ed99c8700a49bbb9ef9d4a13515c17, role_id=ef689f96442945e8a7bc8cbfee13d909)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:29:12.129051 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0b6d8b-29dc-4498-9e6e-70573948d73b admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:29:12.214228 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1244/2496] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:29:12 2022] PUT /identity/v3/projects/cdece90b26c74eb89113d317b1609e92/users/c1ed99c8700a49bbb9ef9d4a13515c17/roles/ef689f96442945e8a7bc8cbfee13d909 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:29:12.219321 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aff094-6b90-4cb0-947d-0fc9677b9203 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:12.219635 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aff094-6b90-4cb0-947d-0fc9677b9203 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:12.219905 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aff094-6b90-4cb0-947d-0fc9677b9203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:12.226911 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:29:12.239742 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-d4aff094-6b90-4cb0-947d-0fc9677b9203 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:29:12.241064 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-d4aff094-6b90-4cb0-947d-0fc9677b9203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:29:12.301435 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1253/2497] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:29:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 00:29:12.310920 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0b6d8b-29dc-4498-9e6e-70573948d73b admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:29:12.312721 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-df0b6d8b-29dc-4498-9e6e-70573948d73b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:29:12.370494 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0b6d8b-29dc-4498-9e6e-70573948d73b admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:29:12.374246 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b17a8df-13a6-40bc-a39a-73d00ebf11f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:29:12.374868 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b17a8df-13a6-40bc-a39a-73d00ebf11f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:12.375139 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b17a8df-13a6-40bc-a39a-73d00ebf11f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:12.375438 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b17a8df-13a6-40bc-a39a-73d00ebf11f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:12.384214 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b17a8df-13a6-40bc-a39a-73d00ebf11f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:29:12.385641 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b17a8df-13a6-40bc-a39a-73d00ebf11f1 service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:29:12.395095 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1245/2498] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:29:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 00:29:13.621660 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6c163b-da15-4c10-80c4-31f71092013d None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:13.622145 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6c163b-da15-4c10-80c4-31f71092013d None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:13.622258 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6c163b-da15-4c10-80c4-31f71092013d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:13.653487 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-df6c163b-da15-4c10-80c4-31f71092013d None None] MFA Rules not processed for user `c1ed99c8700a49bbb9ef9d4a13515c17`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:29:13.655168 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-df6c163b-da15-4c10-80c4-31f71092013d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:29:13.723289 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1254/2499] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:29:13 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:29:13.733889 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b17a8df-13a6-40bc-a39a-73d00ebf11f1 service neutron] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:29:13.735466 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-9b17a8df-13a6-40bc-a39a-73d00ebf11f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:29:13.742352 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:29:13.792482 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b17a8df-13a6-40bc-a39a-73d00ebf11f1 service neutron] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:29:13.795956 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea9f22b-4bb8-4842-a912-6f83251f1f66 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:29:13.796554 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea9f22b-4bb8-4842-a912-6f83251f1f66 service cinder] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:13.796829 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea9f22b-4bb8-4842-a912-6f83251f1f66 service cinder] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:13.797089 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea9f22b-4bb8-4842-a912-6f83251f1f66 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:13.805605 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea9f22b-4bb8-4842-a912-6f83251f1f66 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:29:13.807033 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea9f22b-4bb8-4842-a912-6f83251f1f66 service cinder] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:29:13.817287 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1246/2500] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:29:13 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:29:15.475582 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900bf431-6b0f-4c57-adde-522eb61b09fb None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:15.476102 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900bf431-6b0f-4c57-adde-522eb61b09fb None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:15.476201 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900bf431-6b0f-4c57-adde-522eb61b09fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:15.495401 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-900bf431-6b0f-4c57-adde-522eb61b09fb None None] MFA Rules not processed for user `c1ed99c8700a49bbb9ef9d4a13515c17`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:29:15.496721 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-900bf431-6b0f-4c57-adde-522eb61b09fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:29:15.503508 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1255/2501] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:29:15 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:29:23.812773 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=24 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:29:25.490656 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=61 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:29:30.802426 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea9f22b-4bb8-4842-a912-6f83251f1f66 service cinder] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:29:30.803879 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-6ea9f22b-4bb8-4842-a912-6f83251f1f66 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:29:30.818699 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea9f22b-4bb8-4842-a912-6f83251f1f66 service cinder] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:29:30.823298 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d550639e-d0ca-441e-a2d7-05ed7b1239f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:29:30.823935 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d550639e-d0ca-441e-a2d7-05ed7b1239f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:30.824211 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d550639e-d0ca-441e-a2d7-05ed7b1239f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:30.824473 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d550639e-d0ca-441e-a2d7-05ed7b1239f0 admin admin] PATH_INFO: `/v3/users/257b2e9ebdc14c77995a42164072b298` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:30.825582 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d550639e-d0ca-441e-a2d7-05ed7b1239f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=257b2e9ebdc14c77995a42164072b298)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:29:30.827526 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d550639e-d0ca-441e-a2d7-05ed7b1239f0 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:29:31.034810 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-d550639e-d0ca-441e-a2d7-05ed7b1239f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '257b2e9ebdc14c77995a42164072b298'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:29:31.035595 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-d550639e-d0ca-441e-a2d7-05ed7b1239f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '257b2e9ebdc14c77995a42164072b298'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:29:31.047725 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-d550639e-d0ca-441e-a2d7-05ed7b1239f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '257b2e9ebdc14c77995a42164072b298'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:29:31.111015 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-d550639e-d0ca-441e-a2d7-05ed7b1239f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '257b2e9ebdc14c77995a42164072b298'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:29:31.112303 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-d550639e-d0ca-441e-a2d7-05ed7b1239f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '257b2e9ebdc14c77995a42164072b298'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:29:31.126728 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1247/2502] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:29:30 2022] DELETE /identity/v3/users/257b2e9ebdc14c77995a42164072b298 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:29:31.131313 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3118f061-b94a-4b54-ac59-9405c58da899 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:31.131817 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3118f061-b94a-4b54-ac59-9405c58da899 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:31.131915 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3118f061-b94a-4b54-ac59-9405c58da899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:31.151090 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-3118f061-b94a-4b54-ac59-9405c58da899 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:29:31.152347 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-3118f061-b94a-4b54-ac59-9405c58da899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:29:31.212550 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1256/2503] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:29:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 00:29:31.222649 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d550639e-d0ca-441e-a2d7-05ed7b1239f0 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:29:31.224107 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-d550639e-d0ca-441e-a2d7-05ed7b1239f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:29:31.285266 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d550639e-d0ca-441e-a2d7-05ed7b1239f0 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:29:31.289784 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe72977-37d6-43b5-acd1-6a058904286c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:29:31.290398 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe72977-37d6-43b5-acd1-6a058904286c service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:31.290685 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe72977-37d6-43b5-acd1-6a058904286c service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:31.290959 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe72977-37d6-43b5-acd1-6a058904286c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:31.301189 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe72977-37d6-43b5-acd1-6a058904286c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:29:31.302637 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe72977-37d6-43b5-acd1-6a058904286c service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:29:31.312341 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1248/2504] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:29:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:29:31.707345 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3118f061-b94a-4b54-ac59-9405c58da899 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:29:31.708807 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-3118f061-b94a-4b54-ac59-9405c58da899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:29:31.720822 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3118f061-b94a-4b54-ac59-9405c58da899 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:29:31.726349 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59c7456-0e63-4fed-8123-0e2692b893d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:29:31.727012 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59c7456-0e63-4fed-8123-0e2692b893d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:31.727315 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59c7456-0e63-4fed-8123-0e2692b893d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:31.727649 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59c7456-0e63-4fed-8123-0e2692b893d8 admin admin] PATH_INFO: `/v3/projects/ec802857e5dd4a5cbe5b57fed385196a` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:31.728775 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59c7456-0e63-4fed-8123-0e2692b893d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec802857e5dd4a5cbe5b57fed385196a)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:29:31.730683 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59c7456-0e63-4fed-8123-0e2692b893d8 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:29:31.827535 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-e59c7456-0e63-4fed-8123-0e2692b893d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec802857e5dd4a5cbe5b57fed385196a', 'request_id': 'req-e59c7456-0e63-4fed-8123-0e2692b893d8'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:29:31.835682 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-e59c7456-0e63-4fed-8123-0e2692b893d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec802857e5dd4a5cbe5b57fed385196a', 'request_id': 'req-e59c7456-0e63-4fed-8123-0e2692b893d8'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:29:31.910801 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-e59c7456-0e63-4fed-8123-0e2692b893d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec802857e5dd4a5cbe5b57fed385196a', 'request_id': 'req-e59c7456-0e63-4fed-8123-0e2692b893d8'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:29:31.922017 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-e59c7456-0e63-4fed-8123-0e2692b893d8 admin admin] The token cache is being invalidate because project ec802857e5dd4a5cbe5b57fed385196a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81747) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:29:31.922303 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-e59c7456-0e63-4fed-8123-0e2692b893d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:29:31.922917 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-e59c7456-0e63-4fed-8123-0e2692b893d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:29:31.924480 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1257/2505] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:29:31 2022] DELETE /identity/v3/projects/ec802857e5dd4a5cbe5b57fed385196a => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 00:29:31.929740 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff59116e-e558-4f58-825f-493e44c0f03e None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:31.930224 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff59116e-e558-4f58-825f-493e44c0f03e None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:31.930317 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff59116e-e558-4f58-825f-493e44c0f03e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:31.951088 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-ff59116e-e558-4f58-825f-493e44c0f03e None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:29:31.952315 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-ff59116e-e558-4f58-825f-493e44c0f03e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:29:32.012143 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1249/2506] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:29:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:29:32.020968 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fe8282-09d6-423d-ad00-33e24e39fe6c None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:32.021289 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fe8282-09d6-423d-ad00-33e24e39fe6c None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:32.021546 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fe8282-09d6-423d-ad00-33e24e39fe6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:32.041204 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-25fe8282-09d6-423d-ad00-33e24e39fe6c None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:29:32.042394 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-25fe8282-09d6-423d-ad00-33e24e39fe6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:29:32.050228 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1258/2507] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:29:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:29:32.054452 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff59116e-e558-4f58-825f-493e44c0f03e None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:29:32.064401 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff59116e-e558-4f58-825f-493e44c0f03e None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:29:32.069753 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a238f3-95a5-402e-9ab9-9a1ec8545222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:29:32.070299 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a238f3-95a5-402e-9ab9-9a1ec8545222 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:32.070577 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a238f3-95a5-402e-9ab9-9a1ec8545222 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:32.070844 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a238f3-95a5-402e-9ab9-9a1ec8545222 admin admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:32.071518 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a238f3-95a5-402e-9ab9-9a1ec8545222 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:29:32.071824 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a238f3-95a5-402e-9ab9-9a1ec8545222 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:29:32.073414 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a238f3-95a5-402e-9ab9-9a1ec8545222 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:29:32.085753 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1250/2508] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Oct 12 00:29:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:29:32.090172 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fe8282-09d6-423d-ad00-33e24e39fe6c None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:29:32.099535 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fe8282-09d6-423d-ad00-33e24e39fe6c None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:29:32.104246 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430618a6-c2ac-408c-8d66-86c65ba22e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:29:32.104839 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430618a6-c2ac-408c-8d66-86c65ba22e5a admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:32.105115 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430618a6-c2ac-408c-8d66-86c65ba22e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:32.105383 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430618a6-c2ac-408c-8d66-86c65ba22e5a admin admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:32.106004 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430618a6-c2ac-408c-8d66-86c65ba22e5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:29:32.107923 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430618a6-c2ac-408c-8d66-86c65ba22e5a admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:29:32.221246 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1259/2509] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:29:32 2022] POST /identity/v3/projects => generated 373 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:29:32.228578 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a238f3-95a5-402e-9ab9-9a1ec8545222 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:29:32.233716 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:29:32.240193 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a238f3-95a5-402e-9ab9-9a1ec8545222 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:29:32.301677 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b18bc1-9556-46c6-a1a2-56785e99f2d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:29:32.302291 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b18bc1-9556-46c6-a1a2-56785e99f2d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:32.302597 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b18bc1-9556-46c6-a1a2-56785e99f2d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:32.302876 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b18bc1-9556-46c6-a1a2-56785e99f2d6 admin admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:32.303528 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b18bc1-9556-46c6-a1a2-56785e99f2d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:29:32.305557 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b18bc1-9556-46c6-a1a2-56785e99f2d6 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:29:32.385428 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1251/2510] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:29:32 2022] POST /identity/v3/users => generated 287 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:29:32.389334 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430618a6-c2ac-408c-8d66-86c65ba22e5a admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:29:32.399820 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430618a6-c2ac-408c-8d66-86c65ba22e5a admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:29:32.404409 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de90608-20b2-4f4a-b3e3-1c24254fa69c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:29:32.405004 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de90608-20b2-4f4a-b3e3-1c24254fa69c admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:32.405278 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de90608-20b2-4f4a-b3e3-1c24254fa69c admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:32.405547 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de90608-20b2-4f4a-b3e3-1c24254fa69c admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:32.406173 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de90608-20b2-4f4a-b3e3-1c24254fa69c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:29:32.407946 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de90608-20b2-4f4a-b3e3-1c24254fa69c admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:29:32.418048 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1260/2511] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:29:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:29:32.422251 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b18bc1-9556-46c6-a1a2-56785e99f2d6 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:29:32.432687 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b18bc1-9556-46c6-a1a2-56785e99f2d6 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:29:32.437231 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5849886b-1260-40d4-9672-d046fb5691b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:29:32.437841 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5849886b-1260-40d4-9672-d046fb5691b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:32.438117 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5849886b-1260-40d4-9672-d046fb5691b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:32.438383 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5849886b-1260-40d4-9672-d046fb5691b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:32.439018 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5849886b-1260-40d4-9672-d046fb5691b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:29:32.441298 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5849886b-1260-40d4-9672-d046fb5691b3 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:29:32.451699 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1252/2512] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:29:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:29:32.455701 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de90608-20b2-4f4a-b3e3-1c24254fa69c admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:29:32.465701 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de90608-20b2-4f4a-b3e3-1c24254fa69c admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:29:32.470301 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b6e8a8-129a-4684-8376-626e3db698f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:29:32.472398 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b6e8a8-129a-4684-8376-626e3db698f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:32.472398 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b6e8a8-129a-4684-8376-626e3db698f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:32.472398 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b6e8a8-129a-4684-8376-626e3db698f9 admin admin] PATH_INFO: `/v3/projects/e57b239f6b3b4f41a230945e1e9ef5ad/users/9864ba8be0df4c3e922657b31123d104/roles/ef689f96442945e8a7bc8cbfee13d909` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:32.489102 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b6e8a8-129a-4684-8376-626e3db698f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e57b239f6b3b4f41a230945e1e9ef5ad, user_id=9864ba8be0df4c3e922657b31123d104, role_id=ef689f96442945e8a7bc8cbfee13d909)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:29:32.492159 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b6e8a8-129a-4684-8376-626e3db698f9 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:29:32.574302 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1261/2513] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:29:32 2022] PUT /identity/v3/projects/e57b239f6b3b4f41a230945e1e9ef5ad/users/9864ba8be0df4c3e922657b31123d104/roles/ef689f96442945e8a7bc8cbfee13d909 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:29:32.584710 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a178cbf-710e-4bba-b3fb-a71b9730a02d None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:32.584988 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a178cbf-710e-4bba-b3fb-a71b9730a02d None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:32.586116 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a178cbf-710e-4bba-b3fb-a71b9730a02d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:32.616566 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-5a178cbf-710e-4bba-b3fb-a71b9730a02d None None] MFA Rules not processed for user `9864ba8be0df4c3e922657b31123d104`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:29:32.617864 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-5a178cbf-710e-4bba-b3fb-a71b9730a02d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:29:32.683228 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1253/2514] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:29:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:29:32.694111 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b6e8a8-129a-4684-8376-626e3db698f9 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:29:32.695910 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-88b6e8a8-129a-4684-8376-626e3db698f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:29:32.718403 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:29:32.755832 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b6e8a8-129a-4684-8376-626e3db698f9 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:29:32.760329 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0873ff1b-165a-4ace-a970-c1f69c7eba33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:29:32.760931 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0873ff1b-165a-4ace-a970-c1f69c7eba33 service cinder] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:32.761196 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0873ff1b-165a-4ace-a970-c1f69c7eba33 service cinder] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:32.761466 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0873ff1b-165a-4ace-a970-c1f69c7eba33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:32.770354 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0873ff1b-165a-4ace-a970-c1f69c7eba33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:29:32.771937 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0873ff1b-165a-4ace-a970-c1f69c7eba33 service cinder] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:29:32.780781 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1262/2515] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:29:32 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 00:29:36.096247 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3ca7cc-a145-4893-92f9-af762be174bc None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:36.096731 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3ca7cc-a145-4893-92f9-af762be174bc None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:36.096834 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3ca7cc-a145-4893-92f9-af762be174bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:36.116438 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-ba3ca7cc-a145-4893-92f9-af762be174bc None None] MFA Rules not processed for user `c1ed99c8700a49bbb9ef9d4a13515c17`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:29:36.117728 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-ba3ca7cc-a145-4893-92f9-af762be174bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:29:36.171228 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1254/2516] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:29:36 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:29:36.835018 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0873ff1b-165a-4ace-a970-c1f69c7eba33 service cinder] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:29:36.898659 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0873ff1b-165a-4ace-a970-c1f69c7eba33 service cinder] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:29:36.903295 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a953e20-5584-4bad-b743-f3e4ef6aa376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:29:36.903950 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a953e20-5584-4bad-b743-f3e4ef6aa376 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:36.906132 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a953e20-5584-4bad-b743-f3e4ef6aa376 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:36.906132 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a953e20-5584-4bad-b743-f3e4ef6aa376 admin admin] PATH_INFO: `/v3/users/9864ba8be0df4c3e922657b31123d104` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:36.906132 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a953e20-5584-4bad-b743-f3e4ef6aa376 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9864ba8be0df4c3e922657b31123d104)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:29:36.907526 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a953e20-5584-4bad-b743-f3e4ef6aa376 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:29:37.099753 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-2a953e20-5584-4bad-b743-f3e4ef6aa376 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9864ba8be0df4c3e922657b31123d104'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:29:37.100462 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-2a953e20-5584-4bad-b743-f3e4ef6aa376 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9864ba8be0df4c3e922657b31123d104'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:29:37.116061 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-2a953e20-5584-4bad-b743-f3e4ef6aa376 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9864ba8be0df4c3e922657b31123d104'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:29:37.196250 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-2a953e20-5584-4bad-b743-f3e4ef6aa376 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9864ba8be0df4c3e922657b31123d104'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:29:37.196250 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-2a953e20-5584-4bad-b743-f3e4ef6aa376 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9864ba8be0df4c3e922657b31123d104'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:29:37.206481 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1263/2517] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:29:36 2022] DELETE /identity/v3/users/9864ba8be0df4c3e922657b31123d104 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:29:37.211425 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5a996a-b5c2-48fc-8dc6-4249f3b77a30 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:37.211876 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5a996a-b5c2-48fc-8dc6-4249f3b77a30 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:37.211971 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5a996a-b5c2-48fc-8dc6-4249f3b77a30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:37.231188 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-fc5a996a-b5c2-48fc-8dc6-4249f3b77a30 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:29:37.232429 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-fc5a996a-b5c2-48fc-8dc6-4249f3b77a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:29:37.241851 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:29:37.294563 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1255/2518] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:29:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:29:37.304486 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a953e20-5584-4bad-b743-f3e4ef6aa376 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:29:37.305965 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-2a953e20-5584-4bad-b743-f3e4ef6aa376 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:29:37.351326 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:29:37.363622 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a953e20-5584-4bad-b743-f3e4ef6aa376 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:29:37.368125 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993ef67b-5b05-4be8-a270-8383ea0c91b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:29:37.368125 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993ef67b-5b05-4be8-a270-8383ea0c91b9 service neutron] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:37.368363 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993ef67b-5b05-4be8-a270-8383ea0c91b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:37.368659 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993ef67b-5b05-4be8-a270-8383ea0c91b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:37.377921 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993ef67b-5b05-4be8-a270-8383ea0c91b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:29:37.380216 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993ef67b-5b05-4be8-a270-8383ea0c91b9 service neutron] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:29:37.389776 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1264/2519] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:29:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:29:37.780764 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5a996a-b5c2-48fc-8dc6-4249f3b77a30 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:29:37.782191 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-fc5a996a-b5c2-48fc-8dc6-4249f3b77a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:29:37.795295 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5a996a-b5c2-48fc-8dc6-4249f3b77a30 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:29:37.801067 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc88c6b-10fb-4874-adab-750b34092217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:29:37.801740 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc88c6b-10fb-4874-adab-750b34092217 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:37.802013 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc88c6b-10fb-4874-adab-750b34092217 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:37.802496 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc88c6b-10fb-4874-adab-750b34092217 admin admin] PATH_INFO: `/v3/projects/e57b239f6b3b4f41a230945e1e9ef5ad` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:37.803665 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc88c6b-10fb-4874-adab-750b34092217 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e57b239f6b3b4f41a230945e1e9ef5ad)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:29:37.805626 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc88c6b-10fb-4874-adab-750b34092217 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:29:37.950515 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-abc88c6b-10fb-4874-adab-750b34092217 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e57b239f6b3b4f41a230945e1e9ef5ad', 'request_id': 'req-abc88c6b-10fb-4874-adab-750b34092217'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:29:37.957149 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-abc88c6b-10fb-4874-adab-750b34092217 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e57b239f6b3b4f41a230945e1e9ef5ad', 'request_id': 'req-abc88c6b-10fb-4874-adab-750b34092217'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:29:38.076913 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-abc88c6b-10fb-4874-adab-750b34092217 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e57b239f6b3b4f41a230945e1e9ef5ad', 'request_id': 'req-abc88c6b-10fb-4874-adab-750b34092217'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:29:38.088451 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-abc88c6b-10fb-4874-adab-750b34092217 admin admin] The token cache is being invalidate because project e57b239f6b3b4f41a230945e1e9ef5ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81746) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:29:38.088794 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-abc88c6b-10fb-4874-adab-750b34092217 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:29:38.089455 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-abc88c6b-10fb-4874-adab-750b34092217 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:29:38.090954 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1256/2520] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:29:37 2022] DELETE /identity/v3/projects/e57b239f6b3b4f41a230945e1e9ef5ad => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:29:38.095878 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b270cb5-ba9d-45b8-a6ad-bb1e44bcb09b None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:38.103687 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b270cb5-ba9d-45b8-a6ad-bb1e44bcb09b None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:38.103687 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b270cb5-ba9d-45b8-a6ad-bb1e44bcb09b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:38.117370 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-3b270cb5-ba9d-45b8-a6ad-bb1e44bcb09b None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:29:38.118589 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-3b270cb5-ba9d-45b8-a6ad-bb1e44bcb09b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:29:38.181562 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1265/2521] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:29:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:29:38.189492 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb73348-1d3e-4d67-98f7-d807d8bcbebe None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:38.189844 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb73348-1d3e-4d67-98f7-d807d8bcbebe None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:38.190232 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb73348-1d3e-4d67-98f7-d807d8bcbebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:38.210683 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-ddb73348-1d3e-4d67-98f7-d807d8bcbebe None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:29:38.211924 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-ddb73348-1d3e-4d67-98f7-d807d8bcbebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:29:38.224023 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1257/2522] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:29:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:29:38.228543 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b270cb5-ba9d-45b8-a6ad-bb1e44bcb09b None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:29:38.240252 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b270cb5-ba9d-45b8-a6ad-bb1e44bcb09b None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:29:38.245367 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eadd02-1187-4048-acd9-9eeaf493d9fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:29:38.245984 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eadd02-1187-4048-acd9-9eeaf493d9fa admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:38.246524 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eadd02-1187-4048-acd9-9eeaf493d9fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:38.246751 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eadd02-1187-4048-acd9-9eeaf493d9fa admin admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:38.247449 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eadd02-1187-4048-acd9-9eeaf493d9fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:29:38.247757 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eadd02-1187-4048-acd9-9eeaf493d9fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:29:38.249350 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eadd02-1187-4048-acd9-9eeaf493d9fa admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:29:38.262185 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1266/2523] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Oct 12 00:29:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:29:38.266380 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb73348-1d3e-4d67-98f7-d807d8bcbebe None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:29:38.275857 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb73348-1d3e-4d67-98f7-d807d8bcbebe None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:29:38.280507 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed0dcb4-b8dd-45f8-8bce-3dfd4a65b57e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:29:38.281101 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed0dcb4-b8dd-45f8-8bce-3dfd4a65b57e admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:38.281378 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed0dcb4-b8dd-45f8-8bce-3dfd4a65b57e admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:38.281646 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed0dcb4-b8dd-45f8-8bce-3dfd4a65b57e admin admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:38.282265 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed0dcb4-b8dd-45f8-8bce-3dfd4a65b57e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:29:38.284193 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed0dcb4-b8dd-45f8-8bce-3dfd4a65b57e admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:29:38.358471 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1258/2524] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:29:38 2022] POST /identity/v3/projects => generated 367 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:29:38.361990 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eadd02-1187-4048-acd9-9eeaf493d9fa admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:29:38.372357 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eadd02-1187-4048-acd9-9eeaf493d9fa admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:29:38.429562 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f3c98b-eee6-4b25-807b-b712e964ccc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:29:38.430089 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f3c98b-eee6-4b25-807b-b712e964ccc2 admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:38.430364 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f3c98b-eee6-4b25-807b-b712e964ccc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:38.430634 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f3c98b-eee6-4b25-807b-b712e964ccc2 admin admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:38.431249 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f3c98b-eee6-4b25-807b-b712e964ccc2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:29:38.433202 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f3c98b-eee6-4b25-807b-b712e964ccc2 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:29:38.511253 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1267/2525] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:29:38 2022] POST /identity/v3/users => generated 284 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:29:38.515421 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed0dcb4-b8dd-45f8-8bce-3dfd4a65b57e admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:29:38.526022 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed0dcb4-b8dd-45f8-8bce-3dfd4a65b57e admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:29:38.530604 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7af2ee-3a0e-4bce-ba50-e434b782caab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:29:38.531194 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7af2ee-3a0e-4bce-ba50-e434b782caab admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:38.531504 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7af2ee-3a0e-4bce-ba50-e434b782caab admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:38.531831 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7af2ee-3a0e-4bce-ba50-e434b782caab admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:38.532499 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7af2ee-3a0e-4bce-ba50-e434b782caab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:29:38.534090 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7af2ee-3a0e-4bce-ba50-e434b782caab admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:29:38.548321 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1259/2526] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:29:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:29:38.553699 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f3c98b-eee6-4b25-807b-b712e964ccc2 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:29:38.563731 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f3c98b-eee6-4b25-807b-b712e964ccc2 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:29:38.568384 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfec2e6e-5ab2-4fe9-8a2f-5b4e779a820b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:29:38.568987 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec2e6e-5ab2-4fe9-8a2f-5b4e779a820b admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:38.569263 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec2e6e-5ab2-4fe9-8a2f-5b4e779a820b admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:38.569530 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec2e6e-5ab2-4fe9-8a2f-5b4e779a820b admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:38.570159 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfec2e6e-5ab2-4fe9-8a2f-5b4e779a820b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:29:38.571774 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfec2e6e-5ab2-4fe9-8a2f-5b4e779a820b admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:29:38.585827 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1268/2527] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:29:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:29:38.587110 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7af2ee-3a0e-4bce-ba50-e434b782caab admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:29:38.597880 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7af2ee-3a0e-4bce-ba50-e434b782caab admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:29:38.602815 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbf3c4e-7791-47f3-b49f-7bea9c620b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:29:38.603565 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbf3c4e-7791-47f3-b49f-7bea9c620b55 admin admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:38.603854 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbf3c4e-7791-47f3-b49f-7bea9c620b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:38.604182 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbf3c4e-7791-47f3-b49f-7bea9c620b55 admin admin] PATH_INFO: `/v3/projects/85511ec2474f42719d6cde2067ac7e5e/users/18e3335fea884d77be7940f03563e1e1/roles/ef689f96442945e8a7bc8cbfee13d909` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:38.619541 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbf3c4e-7791-47f3-b49f-7bea9c620b55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85511ec2474f42719d6cde2067ac7e5e, user_id=18e3335fea884d77be7940f03563e1e1, role_id=ef689f96442945e8a7bc8cbfee13d909)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:29:38.622195 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbf3c4e-7791-47f3-b49f-7bea9c620b55 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:29:38.688552 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1260/2528] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:29:38 2022] PUT /identity/v3/projects/85511ec2474f42719d6cde2067ac7e5e/users/18e3335fea884d77be7940f03563e1e1/roles/ef689f96442945e8a7bc8cbfee13d909 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:29:38.699140 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cecaee-ac0e-4a37-a362-b11e3061b96a None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:38.699453 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cecaee-ac0e-4a37-a362-b11e3061b96a None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:38.699841 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cecaee-ac0e-4a37-a362-b11e3061b96a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:38.728177 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-e1cecaee-ac0e-4a37-a362-b11e3061b96a None None] MFA Rules not processed for user `18e3335fea884d77be7940f03563e1e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:29:38.729428 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-e1cecaee-ac0e-4a37-a362-b11e3061b96a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:29:38.769836 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:29:38.794011 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1269/2529] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:29:38 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:29:38.806124 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbf3c4e-7791-47f3-b49f-7bea9c620b55 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:29:38.807932 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-2cbf3c4e-7791-47f3-b49f-7bea9c620b55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:29:38.847426 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:29:38.865106 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbf3c4e-7791-47f3-b49f-7bea9c620b55 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:29:38.868794 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fff08d-8017-4fad-8098-ed016690fbf8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59b883abbb674b3c976bacb64194a66d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:29:38.869427 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fff08d-8017-4fad-8098-ed016690fbf8 service glance] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:29:38.869710 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fff08d-8017-4fad-8098-ed016690fbf8 service glance] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:29:38.870006 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fff08d-8017-4fad-8098-ed016690fbf8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:29:38.878569 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fff08d-8017-4fad-8098-ed016690fbf8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:29:38.880041 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fff08d-8017-4fad-8098-ed016690fbf8 service glance] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:29:38.889407 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1261/2530] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:29:38 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:29:48.836700 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=9 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:29:48.945245 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=10 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:30:03.043938 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494833b0-b30f-4e1a-bb1a-9ad9789150f1 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:30:03.044384 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494833b0-b30f-4e1a-bb1a-9ad9789150f1 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:30:03.044475 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494833b0-b30f-4e1a-bb1a-9ad9789150f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:30:03.065775 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-494833b0-b30f-4e1a-bb1a-9ad9789150f1 None None] MFA Rules not processed for user `d030d18b595144f19d6ad01d4cd5d52b`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:30:03.066983 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-494833b0-b30f-4e1a-bb1a-9ad9789150f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:30:03.123014 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1270/2531] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Oct 12 00:30:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:30:03.276631 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fff08d-8017-4fad-8098-ed016690fbf8 service glance] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:30:03.278035 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-a6fff08d-8017-4fad-8098-ed016690fbf8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:30:03.350210 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fff08d-8017-4fad-8098-ed016690fbf8 service glance] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:30:03.356213 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e39af0-e9af-4d8f-b6ca-f1d3abfaf689 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc1bcffc78844dcbb1a9b7c80285249b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:30:03.356819 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e39af0-e9af-4d8f-b6ca-f1d3abfaf689 service swift] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:30:03.357099 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e39af0-e9af-4d8f-b6ca-f1d3abfaf689 service swift] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:30:03.357370 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e39af0-e9af-4d8f-b6ca-f1d3abfaf689 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:30:03.365397 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e39af0-e9af-4d8f-b6ca-f1d3abfaf689 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:30:03.367057 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e39af0-e9af-4d8f-b6ca-f1d3abfaf689 service swift] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:30:03.375089 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1262/2532] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Oct 12 00:30:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 12 00:30:13.116389 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=25 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:30:13.372194 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=32 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:30:18.181750 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494833b0-b30f-4e1a-bb1a-9ad9789150f1 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:30:18.251144 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494833b0-b30f-4e1a-bb1a-9ad9789150f1 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:30:18.256326 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea1fdf-9e2a-4dfc-ba93-8fdc0bcbaac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:30:18.257068 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea1fdf-9e2a-4dfc-ba93-8fdc0bcbaac1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:30:18.257416 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea1fdf-9e2a-4dfc-ba93-8fdc0bcbaac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:30:18.257759 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea1fdf-9e2a-4dfc-ba93-8fdc0bcbaac1 admin admin] PATH_INFO: `/v3/users/18e3335fea884d77be7940f03563e1e1` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:30:18.259120 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ea1fdf-9e2a-4dfc-ba93-8fdc0bcbaac1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18e3335fea884d77be7940f03563e1e1)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:30:18.261878 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ea1fdf-9e2a-4dfc-ba93-8fdc0bcbaac1 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:30:18.436214 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-45ea1fdf-9e2a-4dfc-ba93-8fdc0bcbaac1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18e3335fea884d77be7940f03563e1e1'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:30:18.436850 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-45ea1fdf-9e2a-4dfc-ba93-8fdc0bcbaac1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18e3335fea884d77be7940f03563e1e1'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:30:18.452654 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-45ea1fdf-9e2a-4dfc-ba93-8fdc0bcbaac1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18e3335fea884d77be7940f03563e1e1'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:30:18.533641 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-45ea1fdf-9e2a-4dfc-ba93-8fdc0bcbaac1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18e3335fea884d77be7940f03563e1e1'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:30:18.534339 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-45ea1fdf-9e2a-4dfc-ba93-8fdc0bcbaac1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18e3335fea884d77be7940f03563e1e1'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:30:18.552335 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1271/2533] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:30:18 2022] DELETE /identity/v3/users/18e3335fea884d77be7940f03563e1e1 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:30:18.557176 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548284df-846c-44c4-9dda-a34024eb6f92 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:30:18.558263 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548284df-846c-44c4-9dda-a34024eb6f92 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:30:18.558263 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548284df-846c-44c4-9dda-a34024eb6f92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:30:18.581222 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-548284df-846c-44c4-9dda-a34024eb6f92 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:30:18.581463 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-548284df-846c-44c4-9dda-a34024eb6f92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:30:18.647009 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1263/2534] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:30:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:30:18.656939 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea1fdf-9e2a-4dfc-ba93-8fdc0bcbaac1 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:30:18.658395 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-45ea1fdf-9e2a-4dfc-ba93-8fdc0bcbaac1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:30:18.725127 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea1fdf-9e2a-4dfc-ba93-8fdc0bcbaac1 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:30:18.728858 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab520e58-12dc-46ad-86e5-92008e5a1bc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:30:18.729490 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab520e58-12dc-46ad-86e5-92008e5a1bc6 service neutron] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:30:18.729762 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab520e58-12dc-46ad-86e5-92008e5a1bc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:30:18.730027 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab520e58-12dc-46ad-86e5-92008e5a1bc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:30:18.740134 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab520e58-12dc-46ad-86e5-92008e5a1bc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:30:18.744068 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab520e58-12dc-46ad-86e5-92008e5a1bc6 service neutron] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:30:18.755973 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1272/2535] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:30:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:30:19.194661 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548284df-846c-44c4-9dda-a34024eb6f92 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:30:19.196150 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-548284df-846c-44c4-9dda-a34024eb6f92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:30:19.235712 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548284df-846c-44c4-9dda-a34024eb6f92 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:30:19.244834 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f243174-4222-4dae-992a-5e20f6a6502f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:30:19.246570 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f243174-4222-4dae-992a-5e20f6a6502f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:30:19.246857 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f243174-4222-4dae-992a-5e20f6a6502f admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:30:19.247196 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f243174-4222-4dae-992a-5e20f6a6502f admin admin] PATH_INFO: `/v3/projects/85511ec2474f42719d6cde2067ac7e5e` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:30:19.248756 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f243174-4222-4dae-992a-5e20f6a6502f admin admin] RBAC: Authorizing `identity:delete_project(project_id=85511ec2474f42719d6cde2067ac7e5e)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:30:19.250977 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f243174-4222-4dae-992a-5e20f6a6502f admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:30:19.369744 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-0f243174-4222-4dae-992a-5e20f6a6502f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85511ec2474f42719d6cde2067ac7e5e', 'request_id': 'req-0f243174-4222-4dae-992a-5e20f6a6502f'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:30:19.379119 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-0f243174-4222-4dae-992a-5e20f6a6502f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85511ec2474f42719d6cde2067ac7e5e', 'request_id': 'req-0f243174-4222-4dae-992a-5e20f6a6502f'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:30:19.459249 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-0f243174-4222-4dae-992a-5e20f6a6502f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85511ec2474f42719d6cde2067ac7e5e', 'request_id': 'req-0f243174-4222-4dae-992a-5e20f6a6502f'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:30:19.474893 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-0f243174-4222-4dae-992a-5e20f6a6502f admin admin] The token cache is being invalidate because project 85511ec2474f42719d6cde2067ac7e5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81746) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:30:19.475207 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-0f243174-4222-4dae-992a-5e20f6a6502f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:30:19.475875 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-0f243174-4222-4dae-992a-5e20f6a6502f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:30:19.477666 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1264/2536] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:30:19 2022] DELETE /identity/v3/projects/85511ec2474f42719d6cde2067ac7e5e => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:30:19.482599 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2153cbfd-aa3c-4ab6-8ddf-a3f9efefb64f None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:30:19.483098 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2153cbfd-aa3c-4ab6-8ddf-a3f9efefb64f None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:30:19.483098 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2153cbfd-aa3c-4ab6-8ddf-a3f9efefb64f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:30:19.503429 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-2153cbfd-aa3c-4ab6-8ddf-a3f9efefb64f None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:30:19.504639 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-2153cbfd-aa3c-4ab6-8ddf-a3f9efefb64f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:30:19.524405 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:30:19.587449 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1273/2537] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:30:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:30:19.604723 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4063c8-06de-4c6e-8ff2-3bfe8259363c None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:30:19.605313 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4063c8-06de-4c6e-8ff2-3bfe8259363c None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:30:19.605313 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4063c8-06de-4c6e-8ff2-3bfe8259363c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:30:19.626286 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-8a4063c8-06de-4c6e-8ff2-3bfe8259363c None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:30:19.627534 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-8a4063c8-06de-4c6e-8ff2-3bfe8259363c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:30:19.636282 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1265/2538] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:30:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:30:19.640552 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2153cbfd-aa3c-4ab6-8ddf-a3f9efefb64f None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:30:19.654723 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2153cbfd-aa3c-4ab6-8ddf-a3f9efefb64f None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:30:19.660177 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd13570-e57d-4514-af6c-852a2093741a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:30:19.660821 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd13570-e57d-4514-af6c-852a2093741a admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:30:19.661115 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd13570-e57d-4514-af6c-852a2093741a admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:30:19.661384 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd13570-e57d-4514-af6c-852a2093741a admin admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:30:19.662327 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd13570-e57d-4514-af6c-852a2093741a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:30:19.663158 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd13570-e57d-4514-af6c-852a2093741a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:30:19.669760 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd13570-e57d-4514-af6c-852a2093741a admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:30:19.683633 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1274/2539] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Oct 12 00:30:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 00:30:19.688201 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4063c8-06de-4c6e-8ff2-3bfe8259363c None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:30:19.697755 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4063c8-06de-4c6e-8ff2-3bfe8259363c None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:30:19.703515 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447b388b-ed54-4b37-a2e6-19b966a5a7a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:30:19.704188 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447b388b-ed54-4b37-a2e6-19b966a5a7a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:30:19.704467 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447b388b-ed54-4b37-a2e6-19b966a5a7a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:30:19.704740 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447b388b-ed54-4b37-a2e6-19b966a5a7a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:30:19.705439 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447b388b-ed54-4b37-a2e6-19b966a5a7a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:30:19.707259 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447b388b-ed54-4b37-a2e6-19b966a5a7a3 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:30:19.792085 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1266/2540] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:30:19 2022] POST /identity/v3/projects => generated 371 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:30:19.796429 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd13570-e57d-4514-af6c-852a2093741a admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:30:19.806382 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd13570-e57d-4514-af6c-852a2093741a admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:30:19.864811 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4c9ac1-80ca-45d1-8a66-f06d688da8fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:30:19.865524 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4c9ac1-80ca-45d1-8a66-f06d688da8fd admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:30:19.865817 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4c9ac1-80ca-45d1-8a66-f06d688da8fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:30:19.866089 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4c9ac1-80ca-45d1-8a66-f06d688da8fd admin admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:30:19.866710 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4c9ac1-80ca-45d1-8a66-f06d688da8fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:30:19.868658 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4c9ac1-80ca-45d1-8a66-f06d688da8fd admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:30:19.956265 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1275/2541] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:30:19 2022] POST /identity/v3/users => generated 286 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 00:30:19.960468 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447b388b-ed54-4b37-a2e6-19b966a5a7a3 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:30:19.970601 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447b388b-ed54-4b37-a2e6-19b966a5a7a3 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:30:19.975197 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8939b6f3-da45-4446-ae54-14ab681cf959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:30:19.975820 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8939b6f3-da45-4446-ae54-14ab681cf959 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:30:19.976093 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8939b6f3-da45-4446-ae54-14ab681cf959 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:30:19.976363 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8939b6f3-da45-4446-ae54-14ab681cf959 admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:30:19.977340 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8939b6f3-da45-4446-ae54-14ab681cf959 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:30:19.978927 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8939b6f3-da45-4446-ae54-14ab681cf959 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:30:19.988810 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1267/2542] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:30:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:30:19.995948 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4c9ac1-80ca-45d1-8a66-f06d688da8fd admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:30:20.005822 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4c9ac1-80ca-45d1-8a66-f06d688da8fd admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:30:20.011155 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14168dc-6ddf-4338-bb29-506af24f71dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:30:20.011803 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14168dc-6ddf-4338-bb29-506af24f71dd admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:30:20.012078 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14168dc-6ddf-4338-bb29-506af24f71dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:30:20.012345 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14168dc-6ddf-4338-bb29-506af24f71dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:30:20.012977 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14168dc-6ddf-4338-bb29-506af24f71dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:30:20.014553 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14168dc-6ddf-4338-bb29-506af24f71dd admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:30:20.025592 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1276/2543] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:30:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:30:20.029829 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8939b6f3-da45-4446-ae54-14ab681cf959 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:30:20.039811 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8939b6f3-da45-4446-ae54-14ab681cf959 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:30:20.044918 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9493e9-0080-4ef9-be19-52c9b979cf23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:30:20.045589 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9493e9-0080-4ef9-be19-52c9b979cf23 admin admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:30:20.045870 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9493e9-0080-4ef9-be19-52c9b979cf23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:30:20.046148 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9493e9-0080-4ef9-be19-52c9b979cf23 admin admin] PATH_INFO: `/v3/projects/b3e2ac16cacb4eee90e5f180a75efa83/users/64d8e31b43f6422dbc720ef2f3013daa/roles/ef689f96442945e8a7bc8cbfee13d909` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:30:20.062163 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9493e9-0080-4ef9-be19-52c9b979cf23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3e2ac16cacb4eee90e5f180a75efa83, user_id=64d8e31b43f6422dbc720ef2f3013daa, role_id=ef689f96442945e8a7bc8cbfee13d909)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:30:20.064841 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9493e9-0080-4ef9-be19-52c9b979cf23 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:30:20.127437 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1268/2544] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:30:20 2022] PUT /identity/v3/projects/b3e2ac16cacb4eee90e5f180a75efa83/users/64d8e31b43f6422dbc720ef2f3013daa/roles/ef689f96442945e8a7bc8cbfee13d909 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:30:20.137553 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d046e-8596-4f2c-a6c2-c1c95b399bc1 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:30:20.137885 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d046e-8596-4f2c-a6c2-c1c95b399bc1 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:30:20.138186 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d046e-8596-4f2c-a6c2-c1c95b399bc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:30:20.165666 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-ce5d046e-8596-4f2c-a6c2-c1c95b399bc1 None None] MFA Rules not processed for user `64d8e31b43f6422dbc720ef2f3013daa`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:30:20.167359 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-ce5d046e-8596-4f2c-a6c2-c1c95b399bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:30:20.230987 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1277/2545] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:30:20 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:30:20.242451 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9493e9-0080-4ef9-be19-52c9b979cf23 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:30:20.244142 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-ec9493e9-0080-4ef9-be19-52c9b979cf23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:30:20.258088 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:30:20.301131 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9493e9-0080-4ef9-be19-52c9b979cf23 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:30:20.304834 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5208d8a2-36c8-4af5-83ff-b49b41419d96 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59b883abbb674b3c976bacb64194a66d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:30:20.305443 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5208d8a2-36c8-4af5-83ff-b49b41419d96 service glance] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:30:20.305716 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5208d8a2-36c8-4af5-83ff-b49b41419d96 service glance] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:30:20.305984 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5208d8a2-36c8-4af5-83ff-b49b41419d96 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:30:20.314691 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5208d8a2-36c8-4af5-83ff-b49b41419d96 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:30:20.316168 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5208d8a2-36c8-4af5-83ff-b49b41419d96 service glance] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:30:20.324781 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1269/2546] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:30:20 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:30:20.829105 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9d4141-80d6-4bd5-8ac5-bf55c73d8cc1 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:30:20.829585 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9d4141-80d6-4bd5-8ac5-bf55c73d8cc1 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:30:20.829682 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9d4141-80d6-4bd5-8ac5-bf55c73d8cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:30:20.848779 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-4f9d4141-80d6-4bd5-8ac5-bf55c73d8cc1 None None] MFA Rules not processed for user `d030d18b595144f19d6ad01d4cd5d52b`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:30:20.849956 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-4f9d4141-80d6-4bd5-8ac5-bf55c73d8cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:30:20.862809 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:30:20.906966 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1278/2547] 10.222.0.36 () {58 vars in 967 bytes} [Wed Oct 12 00:30:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 00:30:20.919564 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5208d8a2-36c8-4af5-83ff-b49b41419d96 service glance] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:30:20.928110 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-5208d8a2-36c8-4af5-83ff-b49b41419d96 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:30:20.990640 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5208d8a2-36c8-4af5-83ff-b49b41419d96 service glance] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:30:20.996323 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cc9f42-f24d-4efa-8326-08066d941cb1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc1bcffc78844dcbb1a9b7c80285249b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:30:20.996928 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cc9f42-f24d-4efa-8326-08066d941cb1 service swift] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:30:20.997202 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cc9f42-f24d-4efa-8326-08066d941cb1 service swift] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:30:21.018175 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cc9f42-f24d-4efa-8326-08066d941cb1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:30:21.018175 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cc9f42-f24d-4efa-8326-08066d941cb1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:30:21.018175 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cc9f42-f24d-4efa-8326-08066d941cb1 service swift] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:30:21.018175 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1270/2548] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Oct 12 00:30:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 12 00:30:23.446876 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9d4141-80d6-4bd5-8ac5-bf55c73d8cc1 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:30:23.448314 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-4f9d4141-80d6-4bd5-8ac5-bf55c73d8cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:30:23.518717 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9d4141-80d6-4bd5-8ac5-bf55c73d8cc1 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:30:23.528023 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a4b3e6-d9e8-4f96-8ae8-fa9cd5d45c0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:30:23.529043 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a4b3e6-d9e8-4f96-8ae8-fa9cd5d45c0a service neutron] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:30:23.529615 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a4b3e6-d9e8-4f96-8ae8-fa9cd5d45c0a service neutron] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:30:23.529892 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a4b3e6-d9e8-4f96-8ae8-fa9cd5d45c0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:30:23.532111 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-92a4b3e6-d9e8-4f96-8ae8-fa9cd5d45c0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:30:23.624402 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a4b3e6-d9e8-4f96-8ae8-fa9cd5d45c0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:30:23.625849 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a4b3e6-d9e8-4f96-8ae8-fa9cd5d45c0a service neutron] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:30:23.646561 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1279/2549] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:30:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:30:23.876696 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cc9f42-f24d-4efa-8326-08066d941cb1 service swift] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:30:23.878102 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-37cc9f42-f24d-4efa-8326-08066d941cb1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:30:23.942395 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cc9f42-f24d-4efa-8326-08066d941cb1 service swift] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:30:23.946799 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5410f9e-10a2-443b-899e-ce34e9e6dfc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:30:23.947454 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5410f9e-10a2-443b-899e-ce34e9e6dfc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:30:23.947732 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5410f9e-10a2-443b-899e-ce34e9e6dfc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:30:23.948000 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5410f9e-10a2-443b-899e-ce34e9e6dfc9 admin admin] PATH_INFO: `/v3/users/c1ed99c8700a49bbb9ef9d4a13515c17` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:30:23.949584 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5410f9e-10a2-443b-899e-ce34e9e6dfc9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1ed99c8700a49bbb9ef9d4a13515c17)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:30:23.951509 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5410f9e-10a2-443b-899e-ce34e9e6dfc9 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:30:24.122343 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:30:24.124006 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-a5410f9e-10a2-443b-899e-ce34e9e6dfc9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1ed99c8700a49bbb9ef9d4a13515c17'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:30:24.124650 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-a5410f9e-10a2-443b-899e-ce34e9e6dfc9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1ed99c8700a49bbb9ef9d4a13515c17'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:30:24.136959 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-a5410f9e-10a2-443b-899e-ce34e9e6dfc9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1ed99c8700a49bbb9ef9d4a13515c17'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:30:24.218549 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-a5410f9e-10a2-443b-899e-ce34e9e6dfc9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1ed99c8700a49bbb9ef9d4a13515c17'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:30:24.219174 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-a5410f9e-10a2-443b-899e-ce34e9e6dfc9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1ed99c8700a49bbb9ef9d4a13515c17'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:30:24.231865 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1271/2550] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:30:23 2022] DELETE /identity/v3/users/c1ed99c8700a49bbb9ef9d4a13515c17 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:30:24.460110 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a4b3e6-d9e8-4f96-8ae8-fa9cd5d45c0a service neutron] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:30:24.471179 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a4b3e6-d9e8-4f96-8ae8-fa9cd5d45c0a service neutron] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:30:24.514141 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:30:24.529161 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dfaabd-c52b-40fa-8293-73ba44ca305a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:30:24.529789 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dfaabd-c52b-40fa-8293-73ba44ca305a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:30:24.530067 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dfaabd-c52b-40fa-8293-73ba44ca305a admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:30:24.530344 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dfaabd-c52b-40fa-8293-73ba44ca305a admin admin] PATH_INFO: `/v3/projects/cdece90b26c74eb89113d317b1609e92` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:30:24.531506 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dfaabd-c52b-40fa-8293-73ba44ca305a admin admin] RBAC: Authorizing `identity:delete_project(project_id=cdece90b26c74eb89113d317b1609e92)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:30:24.533442 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dfaabd-c52b-40fa-8293-73ba44ca305a admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:30:24.670011 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-80dfaabd-c52b-40fa-8293-73ba44ca305a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cdece90b26c74eb89113d317b1609e92', 'request_id': 'req-80dfaabd-c52b-40fa-8293-73ba44ca305a'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:30:24.676547 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-80dfaabd-c52b-40fa-8293-73ba44ca305a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cdece90b26c74eb89113d317b1609e92', 'request_id': 'req-80dfaabd-c52b-40fa-8293-73ba44ca305a'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:30:24.755650 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-80dfaabd-c52b-40fa-8293-73ba44ca305a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cdece90b26c74eb89113d317b1609e92', 'request_id': 'req-80dfaabd-c52b-40fa-8293-73ba44ca305a'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:30:24.766786 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-80dfaabd-c52b-40fa-8293-73ba44ca305a admin admin] The token cache is being invalidate because project cdece90b26c74eb89113d317b1609e92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81747) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:30:24.767077 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-80dfaabd-c52b-40fa-8293-73ba44ca305a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:30:24.767734 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-80dfaabd-c52b-40fa-8293-73ba44ca305a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:30:24.769229 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1280/2551] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:30:24 2022] DELETE /identity/v3/projects/cdece90b26c74eb89113d317b1609e92 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:30:24.847628 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5410f9e-10a2-443b-899e-ce34e9e6dfc9 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:30:24.849122 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-a5410f9e-10a2-443b-899e-ce34e9e6dfc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:30:24.912572 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5410f9e-10a2-443b-899e-ce34e9e6dfc9 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:30:24.965901 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a5fbd4-d723-4279-a7c4-8d8d28a646ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:30:24.966519 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5fbd4-d723-4279-a7c4-8d8d28a646ad service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:30:24.966939 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5fbd4-d723-4279-a7c4-8d8d28a646ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:30:24.967214 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5fbd4-d723-4279-a7c4-8d8d28a646ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:30:24.969132 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-89a5fbd4-d723-4279-a7c4-8d8d28a646ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:30:25.052138 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a5fbd4-d723-4279-a7c4-8d8d28a646ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:30:25.053496 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a5fbd4-d723-4279-a7c4-8d8d28a646ad service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:30:25.065222 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1272/2552] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:30:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:30:33.768847 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dfaabd-c52b-40fa-8293-73ba44ca305a admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:30:33.770506 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-80dfaabd-c52b-40fa-8293-73ba44ca305a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:30:33.783734 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dfaabd-c52b-40fa-8293-73ba44ca305a admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:30:33.787411 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dde1264-d8e2-42fa-976f-3744bd302f4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:30:33.788017 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dde1264-d8e2-42fa-976f-3744bd302f4d service neutron] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:30:33.788294 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dde1264-d8e2-42fa-976f-3744bd302f4d service neutron] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:30:33.788567 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dde1264-d8e2-42fa-976f-3744bd302f4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:30:33.790283 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-1dde1264-d8e2-42fa-976f-3744bd302f4d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:30:33.800562 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dde1264-d8e2-42fa-976f-3744bd302f4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:30:33.802037 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dde1264-d8e2-42fa-976f-3744bd302f4d service neutron] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:30:33.810214 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1281/2553] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:30:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:30:35.060392 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=87,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:30:38.489396 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a5fbd4-d723-4279-a7c4-8d8d28a646ad service neutron] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:30:38.498442 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a5fbd4-d723-4279-a7c4-8d8d28a646ad service neutron] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:30:38.502143 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0110d096-216c-47ee-a2dc-900752d5ac61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:30:38.502750 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0110d096-216c-47ee-a2dc-900752d5ac61 service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:30:38.503028 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0110d096-216c-47ee-a2dc-900752d5ac61 service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:30:38.503297 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0110d096-216c-47ee-a2dc-900752d5ac61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:30:38.505042 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-0110d096-216c-47ee-a2dc-900752d5ac61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:30:38.515291 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0110d096-216c-47ee-a2dc-900752d5ac61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:30:38.516753 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0110d096-216c-47ee-a2dc-900752d5ac61 service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:30:38.525203 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1273/2554] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:30:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:30:38.619894 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dde1264-d8e2-42fa-976f-3744bd302f4d service neutron] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:30:38.628696 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dde1264-d8e2-42fa-976f-3744bd302f4d service neutron] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:30:38.632328 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c13a61-5ae1-4b9d-8454-318d30615de9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:30:38.632923 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c13a61-5ae1-4b9d-8454-318d30615de9 service neutron] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:30:38.633202 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c13a61-5ae1-4b9d-8454-318d30615de9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:30:38.633471 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c13a61-5ae1-4b9d-8454-318d30615de9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:30:38.635165 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-00c13a61-5ae1-4b9d-8454-318d30615de9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:30:38.649349 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c13a61-5ae1-4b9d-8454-318d30615de9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:30:38.650702 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c13a61-5ae1-4b9d-8454-318d30615de9 service neutron] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:30:38.662223 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1282/2555] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:30:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:30:38.721545 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0110d096-216c-47ee-a2dc-900752d5ac61 service neutron] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:30:38.722919 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-0110d096-216c-47ee-a2dc-900752d5ac61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:30:38.790346 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0110d096-216c-47ee-a2dc-900752d5ac61 service neutron] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:30:38.795550 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f424ef4-4f50-43a6-ac2a-fa99a0edc580 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e528da752ed4c78bc747215909f6569', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:30:38.795964 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f424ef4-4f50-43a6-ac2a-fa99a0edc580 service placement] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:30:38.796241 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f424ef4-4f50-43a6-ac2a-fa99a0edc580 service placement] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:30:38.796511 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f424ef4-4f50-43a6-ac2a-fa99a0edc580 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:30:38.798227 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-9f424ef4-4f50-43a6-ac2a-fa99a0edc580 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:30:38.809622 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f424ef4-4f50-43a6-ac2a-fa99a0edc580 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:30:38.811232 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f424ef4-4f50-43a6-ac2a-fa99a0edc580 service placement] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:30:38.821132 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1274/2556] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:30:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:30:48.657425 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=43,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:30:48.816808 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=38 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:30:51.610848 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c13a61-5ae1-4b9d-8454-318d30615de9 service neutron] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:30:51.612313 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-00c13a61-5ae1-4b9d-8454-318d30615de9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:30:51.675795 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c13a61-5ae1-4b9d-8454-318d30615de9 service neutron] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:30:51.680602 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da864d3d-7904-43d3-910d-3dbbbf6bc72c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:30:51.681226 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da864d3d-7904-43d3-910d-3dbbbf6bc72c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:30:51.681502 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da864d3d-7904-43d3-910d-3dbbbf6bc72c admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:30:51.681773 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da864d3d-7904-43d3-910d-3dbbbf6bc72c admin admin] PATH_INFO: `/v3/users/64d8e31b43f6422dbc720ef2f3013daa` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:30:51.682879 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da864d3d-7904-43d3-910d-3dbbbf6bc72c admin admin] RBAC: Authorizing `identity:delete_user(user_id=64d8e31b43f6422dbc720ef2f3013daa)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:30:51.684810 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da864d3d-7904-43d3-910d-3dbbbf6bc72c admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:30:51.821073 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-da864d3d-7904-43d3-910d-3dbbbf6bc72c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64d8e31b43f6422dbc720ef2f3013daa'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:30:51.821723 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-da864d3d-7904-43d3-910d-3dbbbf6bc72c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64d8e31b43f6422dbc720ef2f3013daa'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:30:51.834289 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-da864d3d-7904-43d3-910d-3dbbbf6bc72c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64d8e31b43f6422dbc720ef2f3013daa'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:30:51.949785 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-da864d3d-7904-43d3-910d-3dbbbf6bc72c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64d8e31b43f6422dbc720ef2f3013daa'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:30:51.950394 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-da864d3d-7904-43d3-910d-3dbbbf6bc72c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64d8e31b43f6422dbc720ef2f3013daa'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:30:51.963319 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1283/2557] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:30:51 2022] DELETE /identity/v3/users/64d8e31b43f6422dbc720ef2f3013daa => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:30:51.967864 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67de9de-d0c8-4eb1-a422-1a0117824bc0 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:30:51.968338 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67de9de-d0c8-4eb1-a422-1a0117824bc0 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:30:51.968338 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67de9de-d0c8-4eb1-a422-1a0117824bc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:30:51.986897 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-f67de9de-d0c8-4eb1-a422-1a0117824bc0 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:30:51.988119 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-f67de9de-d0c8-4eb1-a422-1a0117824bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:30:52.044608 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1275/2558] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:30:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:30:52.054557 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da864d3d-7904-43d3-910d-3dbbbf6bc72c admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:30:52.055962 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-da864d3d-7904-43d3-910d-3dbbbf6bc72c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:30:52.112336 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da864d3d-7904-43d3-910d-3dbbbf6bc72c admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:30:52.115898 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fef355-48d0-4bc5-91c3-16f1bca702ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:30:52.116491 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fef355-48d0-4bc5-91c3-16f1bca702ec service neutron] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:30:52.116761 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fef355-48d0-4bc5-91c3-16f1bca702ec service neutron] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:30:52.117025 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fef355-48d0-4bc5-91c3-16f1bca702ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:30:52.125757 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fef355-48d0-4bc5-91c3-16f1bca702ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:30:52.127799 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fef355-48d0-4bc5-91c3-16f1bca702ec service neutron] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:30:52.136914 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1284/2559] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:30:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 00:30:52.534853 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67de9de-d0c8-4eb1-a422-1a0117824bc0 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:30:52.536395 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-f67de9de-d0c8-4eb1-a422-1a0117824bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:30:52.550131 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67de9de-d0c8-4eb1-a422-1a0117824bc0 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:30:52.554914 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e89fe2-4e78-4ae6-a909-4960c45906bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:30:52.555587 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e89fe2-4e78-4ae6-a909-4960c45906bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:30:52.555870 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e89fe2-4e78-4ae6-a909-4960c45906bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:30:52.556147 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e89fe2-4e78-4ae6-a909-4960c45906bb admin admin] PATH_INFO: `/v3/projects/b3e2ac16cacb4eee90e5f180a75efa83` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:30:52.557284 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e89fe2-4e78-4ae6-a909-4960c45906bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3e2ac16cacb4eee90e5f180a75efa83)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:30:52.559200 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e89fe2-4e78-4ae6-a909-4960c45906bb admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:30:52.660534 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-c0e89fe2-4e78-4ae6-a909-4960c45906bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3e2ac16cacb4eee90e5f180a75efa83', 'request_id': 'req-c0e89fe2-4e78-4ae6-a909-4960c45906bb'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:30:52.667570 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-c0e89fe2-4e78-4ae6-a909-4960c45906bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3e2ac16cacb4eee90e5f180a75efa83', 'request_id': 'req-c0e89fe2-4e78-4ae6-a909-4960c45906bb'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:30:52.749980 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-c0e89fe2-4e78-4ae6-a909-4960c45906bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3e2ac16cacb4eee90e5f180a75efa83', 'request_id': 'req-c0e89fe2-4e78-4ae6-a909-4960c45906bb'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:30:52.761635 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-c0e89fe2-4e78-4ae6-a909-4960c45906bb admin admin] The token cache is being invalidate because project b3e2ac16cacb4eee90e5f180a75efa83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81746) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:30:52.761977 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-c0e89fe2-4e78-4ae6-a909-4960c45906bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:30:52.762721 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-c0e89fe2-4e78-4ae6-a909-4960c45906bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:30:52.764237 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1276/2560] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:30:52 2022] DELETE /identity/v3/projects/b3e2ac16cacb4eee90e5f180a75efa83 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:30:52.769155 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a23606-8f9a-4281-a943-b89348295cd0 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:30:52.769639 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a23606-8f9a-4281-a943-b89348295cd0 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:30:52.769734 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a23606-8f9a-4281-a943-b89348295cd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:30:52.789908 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-37a23606-8f9a-4281-a943-b89348295cd0 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:30:52.791024 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-37a23606-8f9a-4281-a943-b89348295cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:30:52.808447 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:30:52.852940 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1285/2561] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:30:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:30:52.916118 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9df025c-a0b3-4822-b1c6-bad1e67263ab None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:30:52.916547 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9df025c-a0b3-4822-b1c6-bad1e67263ab None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:30:52.916920 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9df025c-a0b3-4822-b1c6-bad1e67263ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:30:52.938552 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-d9df025c-a0b3-4822-b1c6-bad1e67263ab None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:30:52.939810 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-d9df025c-a0b3-4822-b1c6-bad1e67263ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:30:52.947736 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1277/2562] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:30:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:30:52.951980 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a23606-8f9a-4281-a943-b89348295cd0 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:30:52.962821 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a23606-8f9a-4281-a943-b89348295cd0 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:30:52.967343 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fd20b2-87eb-47b5-9b61-45ec5f2bf5a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:30:52.967994 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fd20b2-87eb-47b5-9b61-45ec5f2bf5a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:30:52.968268 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fd20b2-87eb-47b5-9b61-45ec5f2bf5a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:30:52.968537 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fd20b2-87eb-47b5-9b61-45ec5f2bf5a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:30:52.969196 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fd20b2-87eb-47b5-9b61-45ec5f2bf5a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:30:52.969495 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fd20b2-87eb-47b5-9b61-45ec5f2bf5a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:30:52.971750 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fd20b2-87eb-47b5-9b61-45ec5f2bf5a4 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:30:52.983149 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1286/2563] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Oct 12 00:30:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 00:30:52.987437 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9df025c-a0b3-4822-b1c6-bad1e67263ab None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:30:52.996948 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9df025c-a0b3-4822-b1c6-bad1e67263ab None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:30:53.001633 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfeaac7-fce5-4c87-b078-5e45ef4e6ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:30:53.002228 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfeaac7-fce5-4c87-b078-5e45ef4e6ce8 admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:30:53.002507 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfeaac7-fce5-4c87-b078-5e45ef4e6ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:30:53.002778 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfeaac7-fce5-4c87-b078-5e45ef4e6ce8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:30:53.003443 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfeaac7-fce5-4c87-b078-5e45ef4e6ce8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:30:53.005347 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfeaac7-fce5-4c87-b078-5e45ef4e6ce8 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:30:53.100225 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1278/2564] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:30:52 2022] POST /identity/v3/projects => generated 363 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:30:53.104449 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fd20b2-87eb-47b5-9b61-45ec5f2bf5a4 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:30:53.114454 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fd20b2-87eb-47b5-9b61-45ec5f2bf5a4 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:30:53.173949 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4261bf-505c-4cfe-9898-0fd73f9afc0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:30:53.174546 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4261bf-505c-4cfe-9898-0fd73f9afc0d admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:30:53.174820 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4261bf-505c-4cfe-9898-0fd73f9afc0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:30:53.175084 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4261bf-505c-4cfe-9898-0fd73f9afc0d admin admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:30:53.175742 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4261bf-505c-4cfe-9898-0fd73f9afc0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:30:53.177647 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4261bf-505c-4cfe-9898-0fd73f9afc0d admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:30:53.235514 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1287/2565] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:30:53 2022] POST /identity/v3/users => generated 282 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:30:53.239630 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfeaac7-fce5-4c87-b078-5e45ef4e6ce8 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:30:53.249688 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfeaac7-fce5-4c87-b078-5e45ef4e6ce8 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:30:53.254694 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c38577a-b760-4d96-875b-f4b76fbe9839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:30:53.255285 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c38577a-b760-4d96-875b-f4b76fbe9839 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:30:53.255777 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c38577a-b760-4d96-875b-f4b76fbe9839 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:30:53.256052 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c38577a-b760-4d96-875b-f4b76fbe9839 admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:30:53.256686 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c38577a-b760-4d96-875b-f4b76fbe9839 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:30:53.258265 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c38577a-b760-4d96-875b-f4b76fbe9839 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:30:53.269462 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1279/2566] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:30:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:30:53.273723 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4261bf-505c-4cfe-9898-0fd73f9afc0d admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:30:53.283622 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4261bf-505c-4cfe-9898-0fd73f9afc0d admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:30:53.288556 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4184a7b-5b64-45b4-a50c-7f5c185156ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:30:53.289152 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4184a7b-5b64-45b4-a50c-7f5c185156ad admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:30:53.289429 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4184a7b-5b64-45b4-a50c-7f5c185156ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:30:53.289700 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4184a7b-5b64-45b4-a50c-7f5c185156ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:30:53.290362 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4184a7b-5b64-45b4-a50c-7f5c185156ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:30:53.291986 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4184a7b-5b64-45b4-a50c-7f5c185156ad admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:30:53.302609 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1288/2567] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:30:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 00:30:53.306530 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c38577a-b760-4d96-875b-f4b76fbe9839 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:30:53.316914 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c38577a-b760-4d96-875b-f4b76fbe9839 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:30:53.321456 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0489f167-fb6a-4126-a3b2-c5f007d0a91b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:30:53.322089 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0489f167-fb6a-4126-a3b2-c5f007d0a91b admin admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:30:53.322364 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0489f167-fb6a-4126-a3b2-c5f007d0a91b admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:30:53.322635 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0489f167-fb6a-4126-a3b2-c5f007d0a91b admin admin] PATH_INFO: `/v3/projects/8b8bb0f4706544218dd37d7c05938a87/users/b3b07bce17024b7baeb649b81798e920/roles/ef689f96442945e8a7bc8cbfee13d909` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:30:53.337563 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0489f167-fb6a-4126-a3b2-c5f007d0a91b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b8bb0f4706544218dd37d7c05938a87, user_id=b3b07bce17024b7baeb649b81798e920, role_id=ef689f96442945e8a7bc8cbfee13d909)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:30:53.340215 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0489f167-fb6a-4126-a3b2-c5f007d0a91b admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:30:53.400886 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1280/2568] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:30:53 2022] PUT /identity/v3/projects/8b8bb0f4706544218dd37d7c05938a87/users/b3b07bce17024b7baeb649b81798e920/roles/ef689f96442945e8a7bc8cbfee13d909 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:30:53.415807 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff3048e-dadc-4de1-8cf6-e8075f0a49f5 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:30:53.416087 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff3048e-dadc-4de1-8cf6-e8075f0a49f5 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:30:53.416354 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff3048e-dadc-4de1-8cf6-e8075f0a49f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:30:53.444837 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-7ff3048e-dadc-4de1-8cf6-e8075f0a49f5 None None] MFA Rules not processed for user `b3b07bce17024b7baeb649b81798e920`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:30:53.446048 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-7ff3048e-dadc-4de1-8cf6-e8075f0a49f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:30:53.509242 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1289/2569] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:30:53 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:30:53.521558 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0489f167-fb6a-4126-a3b2-c5f007d0a91b admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:30:53.523139 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-0489f167-fb6a-4126-a3b2-c5f007d0a91b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:30:53.545009 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:30:53.581009 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0489f167-fb6a-4126-a3b2-c5f007d0a91b admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:30:53.584619 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1123b36c-3e4e-4b97-85b7-38312290ed34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:30:53.589420 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1123b36c-3e4e-4b97-85b7-38312290ed34 service cinder] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:30:53.589420 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1123b36c-3e4e-4b97-85b7-38312290ed34 service cinder] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:30:53.589420 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1123b36c-3e4e-4b97-85b7-38312290ed34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:30:53.593760 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1123b36c-3e4e-4b97-85b7-38312290ed34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:30:53.595185 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1123b36c-3e4e-4b97-85b7-38312290ed34 service cinder] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:30:53.604559 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1281/2570] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:30:53 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:30:57.494263 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff3048e-dadc-4de1-8cf6-e8075f0a49f5 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:30:57.495809 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-7ff3048e-dadc-4de1-8cf6-e8075f0a49f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:30:57.515347 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:30:57.569412 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff3048e-dadc-4de1-8cf6-e8075f0a49f5 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:30:57.574818 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb1d61d-f47a-4a57-8c28-925e1777ab16 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a515b75aa3944912ad2364ffd7924f40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:30:57.575460 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb1d61d-f47a-4a57-8c28-925e1777ab16 service nova] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:30:57.575739 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb1d61d-f47a-4a57-8c28-925e1777ab16 service nova] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:30:57.576007 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb1d61d-f47a-4a57-8c28-925e1777ab16 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:30:57.577687 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-7eb1d61d-f47a-4a57-8c28-925e1777ab16 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:30:57.590394 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb1d61d-f47a-4a57-8c28-925e1777ab16 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:30:57.592030 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb1d61d-f47a-4a57-8c28-925e1777ab16 service nova] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:30:57.603597 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1290/2571] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:30:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:31:03.629674 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=17 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:31:07.744249 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=26 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:31:33.907193 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1123b36c-3e4e-4b97-85b7-38312290ed34 service cinder] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:31:33.968323 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1123b36c-3e4e-4b97-85b7-38312290ed34 service cinder] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:31:33.973008 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7167bf0-923a-40aa-9ba4-1b880b086ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:31:33.973635 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7167bf0-923a-40aa-9ba4-1b880b086ab4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:31:33.973941 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7167bf0-923a-40aa-9ba4-1b880b086ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:31:33.974211 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7167bf0-923a-40aa-9ba4-1b880b086ab4 admin admin] PATH_INFO: `/v3/users/b3b07bce17024b7baeb649b81798e920` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:31:33.975357 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7167bf0-923a-40aa-9ba4-1b880b086ab4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3b07bce17024b7baeb649b81798e920)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:31:33.977973 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7167bf0-923a-40aa-9ba4-1b880b086ab4 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:31:34.133821 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-c7167bf0-923a-40aa-9ba4-1b880b086ab4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3b07bce17024b7baeb649b81798e920'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:31:34.134468 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-c7167bf0-923a-40aa-9ba4-1b880b086ab4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3b07bce17024b7baeb649b81798e920'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:31:34.146800 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-c7167bf0-923a-40aa-9ba4-1b880b086ab4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3b07bce17024b7baeb649b81798e920'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:31:34.206351 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-c7167bf0-923a-40aa-9ba4-1b880b086ab4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3b07bce17024b7baeb649b81798e920'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:31:34.206971 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-c7167bf0-923a-40aa-9ba4-1b880b086ab4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3b07bce17024b7baeb649b81798e920'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:31:34.220035 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1282/2572] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:31:33 2022] DELETE /identity/v3/users/b3b07bce17024b7baeb649b81798e920 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:31:34.224742 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977aa35e-5d2b-4a76-bab9-91ec2ae19412 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:31:34.225278 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977aa35e-5d2b-4a76-bab9-91ec2ae19412 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:31:34.225278 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977aa35e-5d2b-4a76-bab9-91ec2ae19412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:31:34.244712 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-977aa35e-5d2b-4a76-bab9-91ec2ae19412 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:31:34.245922 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-977aa35e-5d2b-4a76-bab9-91ec2ae19412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:31:34.303473 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1291/2573] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:31:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:31:34.313261 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7167bf0-923a-40aa-9ba4-1b880b086ab4 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:31:34.314666 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-c7167bf0-923a-40aa-9ba4-1b880b086ab4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:31:34.371149 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7167bf0-923a-40aa-9ba4-1b880b086ab4 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:31:34.375062 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3835ec19-0466-44c7-b206-5baf0f0e75d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:31:34.375700 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3835ec19-0466-44c7-b206-5baf0f0e75d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:31:34.375979 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3835ec19-0466-44c7-b206-5baf0f0e75d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:31:34.376249 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3835ec19-0466-44c7-b206-5baf0f0e75d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:31:34.385844 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3835ec19-0466-44c7-b206-5baf0f0e75d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:31:34.387219 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3835ec19-0466-44c7-b206-5baf0f0e75d4 service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:31:34.397096 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1283/2574] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:31:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:31:34.745627 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977aa35e-5d2b-4a76-bab9-91ec2ae19412 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:31:34.751948 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-977aa35e-5d2b-4a76-bab9-91ec2ae19412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:31:34.761327 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977aa35e-5d2b-4a76-bab9-91ec2ae19412 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:31:34.766661 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a30b88-7907-43e7-92d6-6bc380ed13bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:31:34.772237 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a30b88-7907-43e7-92d6-6bc380ed13bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:31:34.772237 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a30b88-7907-43e7-92d6-6bc380ed13bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:31:34.772237 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a30b88-7907-43e7-92d6-6bc380ed13bc admin admin] PATH_INFO: `/v3/projects/8b8bb0f4706544218dd37d7c05938a87` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:31:34.772237 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a30b88-7907-43e7-92d6-6bc380ed13bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b8bb0f4706544218dd37d7c05938a87)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:31:34.772237 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a30b88-7907-43e7-92d6-6bc380ed13bc admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:31:34.893351 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-94a30b88-7907-43e7-92d6-6bc380ed13bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b8bb0f4706544218dd37d7c05938a87', 'request_id': 'req-94a30b88-7907-43e7-92d6-6bc380ed13bc'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:31:34.900035 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-94a30b88-7907-43e7-92d6-6bc380ed13bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b8bb0f4706544218dd37d7c05938a87', 'request_id': 'req-94a30b88-7907-43e7-92d6-6bc380ed13bc'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:31:34.983634 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-94a30b88-7907-43e7-92d6-6bc380ed13bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b8bb0f4706544218dd37d7c05938a87', 'request_id': 'req-94a30b88-7907-43e7-92d6-6bc380ed13bc'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:31:34.994856 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-94a30b88-7907-43e7-92d6-6bc380ed13bc admin admin] The token cache is being invalidate because project 8b8bb0f4706544218dd37d7c05938a87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81747) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:31:34.997263 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-94a30b88-7907-43e7-92d6-6bc380ed13bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:31:34.997263 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-94a30b88-7907-43e7-92d6-6bc380ed13bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:31:34.997263 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1292/2575] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:31:34 2022] DELETE /identity/v3/projects/8b8bb0f4706544218dd37d7c05938a87 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:31:35.002355 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70fcade-eef5-4ee3-a09e-439e092e557c None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:31:35.002860 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70fcade-eef5-4ee3-a09e-439e092e557c None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:31:35.002958 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70fcade-eef5-4ee3-a09e-439e092e557c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:31:35.022856 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-d70fcade-eef5-4ee3-a09e-439e092e557c None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:31:35.024111 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-d70fcade-eef5-4ee3-a09e-439e092e557c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:31:35.041350 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:31:35.082520 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1284/2576] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:31:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:31:35.091800 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7894976c-a572-43c3-830c-25dda3e41a90 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:31:35.092086 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7894976c-a572-43c3-830c-25dda3e41a90 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:31:35.092353 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7894976c-a572-43c3-830c-25dda3e41a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:31:35.112567 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-7894976c-a572-43c3-830c-25dda3e41a90 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:31:35.113782 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-7894976c-a572-43c3-830c-25dda3e41a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:31:35.122367 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1293/2577] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:31:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:31:35.126787 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70fcade-eef5-4ee3-a09e-439e092e557c None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:31:35.136715 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70fcade-eef5-4ee3-a09e-439e092e557c None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:31:35.141363 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80c8d16-e399-481f-b131-324fd7ab7415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:31:35.141991 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80c8d16-e399-481f-b131-324fd7ab7415 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:31:35.142268 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80c8d16-e399-481f-b131-324fd7ab7415 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:31:35.142584 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80c8d16-e399-481f-b131-324fd7ab7415 admin admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:31:35.143226 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80c8d16-e399-481f-b131-324fd7ab7415 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 00:31:35.143563 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80c8d16-e399-481f-b131-324fd7ab7415 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:31:35.145180 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80c8d16-e399-481f-b131-324fd7ab7415 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:31:35.157280 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1285/2578] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Oct 12 00:31:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 00:31:35.161393 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7894976c-a572-43c3-830c-25dda3e41a90 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:31:35.171297 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7894976c-a572-43c3-830c-25dda3e41a90 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:31:35.176108 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ea3359-377b-4f09-af36-1c3c7644b422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:31:35.176708 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ea3359-377b-4f09-af36-1c3c7644b422 admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:31:35.176987 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ea3359-377b-4f09-af36-1c3c7644b422 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:31:35.177260 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ea3359-377b-4f09-af36-1c3c7644b422 admin admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:31:35.177895 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ea3359-377b-4f09-af36-1c3c7644b422 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:31:35.179816 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ea3359-377b-4f09-af36-1c3c7644b422 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:31:35.253402 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1294/2579] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:31:35 2022] POST /identity/v3/projects => generated 379 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 00:31:35.257617 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80c8d16-e399-481f-b131-324fd7ab7415 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:31:35.267910 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80c8d16-e399-481f-b131-324fd7ab7415 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:31:35.323234 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f73184d-7d8a-4592-93a0-e2b66bf95058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:31:35.323869 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f73184d-7d8a-4592-93a0-e2b66bf95058 admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:31:35.324145 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f73184d-7d8a-4592-93a0-e2b66bf95058 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:31:35.324417 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f73184d-7d8a-4592-93a0-e2b66bf95058 admin admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:31:35.325036 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f73184d-7d8a-4592-93a0-e2b66bf95058 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:31:35.326947 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f73184d-7d8a-4592-93a0-e2b66bf95058 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:31:35.407337 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1286/2580] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:31:35 2022] POST /identity/v3/users => generated 290 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:31:35.411481 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ea3359-377b-4f09-af36-1c3c7644b422 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:31:35.422519 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ea3359-377b-4f09-af36-1c3c7644b422 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:31:35.427329 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28002c7-b653-4592-9237-5c69fafad6bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:31:35.428059 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28002c7-b653-4592-9237-5c69fafad6bb admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:31:35.428367 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28002c7-b653-4592-9237-5c69fafad6bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:31:35.428655 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28002c7-b653-4592-9237-5c69fafad6bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:31:35.429292 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28002c7-b653-4592-9237-5c69fafad6bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:31:35.430878 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28002c7-b653-4592-9237-5c69fafad6bb admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:31:35.440756 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1295/2581] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:31:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:31:35.444497 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f73184d-7d8a-4592-93a0-e2b66bf95058 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:31:35.454269 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f73184d-7d8a-4592-93a0-e2b66bf95058 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:31:35.458856 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2a7546-b02a-463c-806a-1c13086bf1ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:31:35.459478 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2a7546-b02a-463c-806a-1c13086bf1ac admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:31:35.459760 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2a7546-b02a-463c-806a-1c13086bf1ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:31:35.460031 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2a7546-b02a-463c-806a-1c13086bf1ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:31:35.460662 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2a7546-b02a-463c-806a-1c13086bf1ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:31:35.462251 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2a7546-b02a-463c-806a-1c13086bf1ac admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:31:35.473129 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1287/2582] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:31:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:31:35.476957 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28002c7-b653-4592-9237-5c69fafad6bb admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:31:35.487306 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28002c7-b653-4592-9237-5c69fafad6bb admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:31:35.491935 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e621d026-1386-4922-a6d3-e03e5e4090c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:31:35.492577 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e621d026-1386-4922-a6d3-e03e5e4090c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:31:35.492858 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e621d026-1386-4922-a6d3-e03e5e4090c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:31:35.493131 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e621d026-1386-4922-a6d3-e03e5e4090c2 admin admin] PATH_INFO: `/v3/projects/764ed0f4ff5d4c48b89dc4be9460323b/users/828b7770373f4e9688b0f218f7fd6e4f/roles/ef689f96442945e8a7bc8cbfee13d909` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:31:35.507542 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e621d026-1386-4922-a6d3-e03e5e4090c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=764ed0f4ff5d4c48b89dc4be9460323b, user_id=828b7770373f4e9688b0f218f7fd6e4f, role_id=ef689f96442945e8a7bc8cbfee13d909)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:31:35.510145 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e621d026-1386-4922-a6d3-e03e5e4090c2 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:31:35.584383 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1296/2583] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:31:35 2022] PUT /identity/v3/projects/764ed0f4ff5d4c48b89dc4be9460323b/users/828b7770373f4e9688b0f218f7fd6e4f/roles/ef689f96442945e8a7bc8cbfee13d909 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 00:31:35.589927 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0809a7b4-242f-4544-a987-0502e8f64292 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:31:35.590198 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0809a7b4-242f-4544-a987-0502e8f64292 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:31:35.590474 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0809a7b4-242f-4544-a987-0502e8f64292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:31:35.609762 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-0809a7b4-242f-4544-a987-0502e8f64292 None None] MFA Rules not processed for user `1fcb9022bfe743ae9405fc8470a5e016`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:31:35.610957 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-0809a7b4-242f-4544-a987-0502e8f64292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:31:35.669280 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1288/2584] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:31:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:31:35.678962 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e621d026-1386-4922-a6d3-e03e5e4090c2 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:31:35.680731 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-e621d026-1386-4922-a6d3-e03e5e4090c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:31:35.702988 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:31:35.737749 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e621d026-1386-4922-a6d3-e03e5e4090c2 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:31:35.741465 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a7a15f-e150-4dd2-bdb4-92be85b0a6f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:31:35.742068 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a7a15f-e150-4dd2-bdb4-92be85b0a6f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:31:35.742380 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a7a15f-e150-4dd2-bdb4-92be85b0a6f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:31:35.742673 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a7a15f-e150-4dd2-bdb4-92be85b0a6f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:31:35.752136 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a7a15f-e150-4dd2-bdb4-92be85b0a6f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:31:35.753648 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a7a15f-e150-4dd2-bdb4-92be85b0a6f4 service neutron] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:31:35.763651 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1297/2585] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:31:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:31:40.694309 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9a3a4d-4501-4cc2-9d61-a51bcbe8a191 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:31:40.694844 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9a3a4d-4501-4cc2-9d61-a51bcbe8a191 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:31:40.694844 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9a3a4d-4501-4cc2-9d61-a51bcbe8a191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:31:40.727342 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-4a9a3a4d-4501-4cc2-9d61-a51bcbe8a191 None None] MFA Rules not processed for user `828b7770373f4e9688b0f218f7fd6e4f`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:31:40.728591 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-4a9a3a4d-4501-4cc2-9d61-a51bcbe8a191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:31:40.748409 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:31:40.798802 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1289/2586] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:31:40 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:31:40.803503 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a7a15f-e150-4dd2-bdb4-92be85b0a6f4 service neutron] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:31:40.814132 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a7a15f-e150-4dd2-bdb4-92be85b0a6f4 service neutron] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:31:40.819262 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e70ec73-6e31-47a1-822e-99f7742d8ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:31:40.820185 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e70ec73-6e31-47a1-822e-99f7742d8ff8 admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:31:40.820574 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e70ec73-6e31-47a1-822e-99f7742d8ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:31:40.820689 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e70ec73-6e31-47a1-822e-99f7742d8ff8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:31:40.821279 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e70ec73-6e31-47a1-822e-99f7742d8ff8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:31:40.823887 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e70ec73-6e31-47a1-822e-99f7742d8ff8 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:31:40.906013 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1298/2587] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Oct 12 00:31:40 2022] POST /identity/v3/projects => generated 379 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 00:31:40.910286 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9a3a4d-4501-4cc2-9d61-a51bcbe8a191 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:31:40.922354 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9a3a4d-4501-4cc2-9d61-a51bcbe8a191 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:31:40.981160 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f1fdc1-c87e-4f39-a004-df04f5b73607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:31:40.981775 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f1fdc1-c87e-4f39-a004-df04f5b73607 admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:31:40.982049 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f1fdc1-c87e-4f39-a004-df04f5b73607 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:31:40.982315 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f1fdc1-c87e-4f39-a004-df04f5b73607 admin admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:31:40.982943 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f1fdc1-c87e-4f39-a004-df04f5b73607 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:31:40.985014 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f1fdc1-c87e-4f39-a004-df04f5b73607 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:31:41.070267 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1290/2588] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Oct 12 00:31:40 2022] POST /identity/v3/users => generated 296 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 00:31:41.074248 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e70ec73-6e31-47a1-822e-99f7742d8ff8 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:31:41.084659 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e70ec73-6e31-47a1-822e-99f7742d8ff8 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:31:41.089278 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aa5984-a17d-4818-9fe2-8d4a81d44b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:31:41.089873 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aa5984-a17d-4818-9fe2-8d4a81d44b33 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:31:41.090151 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aa5984-a17d-4818-9fe2-8d4a81d44b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:31:41.090418 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aa5984-a17d-4818-9fe2-8d4a81d44b33 admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:31:41.091047 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94aa5984-a17d-4818-9fe2-8d4a81d44b33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:31:41.092689 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94aa5984-a17d-4818-9fe2-8d4a81d44b33 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:31:41.103661 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1299/2589] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:31:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:31:41.107712 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f1fdc1-c87e-4f39-a004-df04f5b73607 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:31:41.117652 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f1fdc1-c87e-4f39-a004-df04f5b73607 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:31:41.122779 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d7e723-ee22-4c27-aa0e-462bafc18140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:31:41.123459 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d7e723-ee22-4c27-aa0e-462bafc18140 admin admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:31:41.123735 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d7e723-ee22-4c27-aa0e-462bafc18140 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:31:41.124005 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d7e723-ee22-4c27-aa0e-462bafc18140 admin admin] PATH_INFO: `/v3/projects/9082ff8249a9467abcfb4708046a18c6/users/306f020dfd444edba42f023d742cd1c3/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:31:41.138227 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d7e723-ee22-4c27-aa0e-462bafc18140 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9082ff8249a9467abcfb4708046a18c6, user_id=306f020dfd444edba42f023d742cd1c3, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:31:41.140957 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d7e723-ee22-4c27-aa0e-462bafc18140 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:31:41.236203 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1291/2590] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:31:41 2022] PUT /identity/v3/projects/9082ff8249a9467abcfb4708046a18c6/users/306f020dfd444edba42f023d742cd1c3/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:31:41.240092 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aa5984-a17d-4818-9fe2-8d4a81d44b33 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:31:41.250723 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aa5984-a17d-4818-9fe2-8d4a81d44b33 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:31:41.305891 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2daa5e3-5563-4c8e-8bdc-9facfa9148f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:31:41.307130 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2daa5e3-5563-4c8e-8bdc-9facfa9148f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:31:41.307439 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2daa5e3-5563-4c8e-8bdc-9facfa9148f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:31:41.307707 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2daa5e3-5563-4c8e-8bdc-9facfa9148f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:31:41.308339 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2daa5e3-5563-4c8e-8bdc-9facfa9148f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:31:41.309922 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2daa5e3-5563-4c8e-8bdc-9facfa9148f9 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:31:41.319984 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1300/2591] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Oct 12 00:31:41 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 00:31:41.324127 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d7e723-ee22-4c27-aa0e-462bafc18140 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:31:41.334496 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d7e723-ee22-4c27-aa0e-462bafc18140 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:31:41.339664 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac20a80-9aa1-461e-9e34-1add69845253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:31:41.340311 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac20a80-9aa1-461e-9e34-1add69845253 admin admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:31:41.340587 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac20a80-9aa1-461e-9e34-1add69845253 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:31:41.340858 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac20a80-9aa1-461e-9e34-1add69845253 admin admin] PATH_INFO: `/v3/projects/9082ff8249a9467abcfb4708046a18c6/users/306f020dfd444edba42f023d742cd1c3/roles/b2e41f0d7a304dc7af8b6a232049b6a2` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:31:41.342788 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac20a80-9aa1-461e-9e34-1add69845253 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9082ff8249a9467abcfb4708046a18c6, user_id=306f020dfd444edba42f023d742cd1c3, role_id=b2e41f0d7a304dc7af8b6a232049b6a2)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:31:41.345440 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac20a80-9aa1-461e-9e34-1add69845253 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:31:41.356135 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1292/2592] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Oct 12 00:31:41 2022] PUT /identity/v3/projects/9082ff8249a9467abcfb4708046a18c6/users/306f020dfd444edba42f023d742cd1c3/roles/b2e41f0d7a304dc7af8b6a232049b6a2 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:31:46.620009 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07427ff6-54b1-430f-8a0e-790762132e5c None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:31:46.620554 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07427ff6-54b1-430f-8a0e-790762132e5c None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:31:46.620554 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07427ff6-54b1-430f-8a0e-790762132e5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:31:46.651814 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-07427ff6-54b1-430f-8a0e-790762132e5c None None] MFA Rules not processed for user `306f020dfd444edba42f023d742cd1c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:31:46.653048 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-07427ff6-54b1-430f-8a0e-790762132e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:31:46.726049 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1301/2593] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:31:46 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:31:46.741869 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac20a80-9aa1-461e-9e34-1add69845253 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:31:46.743767 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-dac20a80-9aa1-461e-9e34-1add69845253 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:31:46.825365 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac20a80-9aa1-461e-9e34-1add69845253 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:31:46.831102 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92062b91-01d3-4110-84e1-3dfc3e8f87f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a515b75aa3944912ad2364ffd7924f40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:31:46.831691 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92062b91-01d3-4110-84e1-3dfc3e8f87f4 service nova] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:31:46.831963 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92062b91-01d3-4110-84e1-3dfc3e8f87f4 service nova] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:31:46.832230 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92062b91-01d3-4110-84e1-3dfc3e8f87f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:31:46.839544 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:31:46.893686 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92062b91-01d3-4110-84e1-3dfc3e8f87f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:31:46.895311 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92062b91-01d3-4110-84e1-3dfc3e8f87f4 service nova] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:31:46.905604 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1293/2594] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:31:46 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:31:56.722844 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=94,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:31:56.955094 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=24 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:32:08.022323 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf487823-70ad-4942-ba60-a04b3b656343 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:32:08.022852 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf487823-70ad-4942-ba60-a04b3b656343 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:32:08.022852 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf487823-70ad-4942-ba60-a04b3b656343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:32:08.041978 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-bf487823-70ad-4942-ba60-a04b3b656343 None None] MFA Rules not processed for user `828b7770373f4e9688b0f218f7fd6e4f`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:32:08.043174 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-bf487823-70ad-4942-ba60-a04b3b656343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:32:08.050167 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1302/2595] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:32:08 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:32:08.362309 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92062b91-01d3-4110-84e1-3dfc3e8f87f4 service nova] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:32:08.363817 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-92062b91-01d3-4110-84e1-3dfc3e8f87f4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:32:08.419910 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92062b91-01d3-4110-84e1-3dfc3e8f87f4 service nova] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:32:08.423552 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2ef266-9dcf-4af7-b236-858841d6980b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:32:08.424208 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2ef266-9dcf-4af7-b236-858841d6980b service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:32:08.424439 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2ef266-9dcf-4af7-b236-858841d6980b service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:32:08.424712 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2ef266-9dcf-4af7-b236-858841d6980b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:32:08.426394 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-eb2ef266-9dcf-4af7-b236-858841d6980b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:32:08.436216 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2ef266-9dcf-4af7-b236-858841d6980b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:32:08.437639 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2ef266-9dcf-4af7-b236-858841d6980b service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:32:08.445704 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1294/2596] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:32:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:32:12.806066 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf487823-70ad-4942-ba60-a04b3b656343 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:32:12.819999 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf487823-70ad-4942-ba60-a04b3b656343 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:32:12.825617 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef886332-0233-4fa6-97be-2d6f3d4d69ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a515b75aa3944912ad2364ffd7924f40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:32:12.826229 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef886332-0233-4fa6-97be-2d6f3d4d69ba service nova] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:32:12.826507 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef886332-0233-4fa6-97be-2d6f3d4d69ba service nova] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:32:12.826779 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef886332-0233-4fa6-97be-2d6f3d4d69ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:32:12.828750 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-ef886332-0233-4fa6-97be-2d6f3d4d69ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:32:12.842471 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef886332-0233-4fa6-97be-2d6f3d4d69ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:32:12.844083 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef886332-0233-4fa6-97be-2d6f3d4d69ba service nova] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:32:12.858824 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1303/2597] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:32:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:32:18.441942 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=26 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:32:20.937902 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2ef266-9dcf-4af7-b236-858841d6980b service neutron] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:32:20.939956 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-eb2ef266-9dcf-4af7-b236-858841d6980b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:32:20.957036 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2ef266-9dcf-4af7-b236-858841d6980b service neutron] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:32:20.962742 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287d5573-48da-40ce-bd19-58ed1043941b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a515b75aa3944912ad2364ffd7924f40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:32:20.963456 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d5573-48da-40ce-bd19-58ed1043941b service nova] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:32:20.963867 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d5573-48da-40ce-bd19-58ed1043941b service nova] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:32:20.964170 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d5573-48da-40ce-bd19-58ed1043941b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:32:20.966071 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-287d5573-48da-40ce-bd19-58ed1043941b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:32:20.979408 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287d5573-48da-40ce-bd19-58ed1043941b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:32:20.980995 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287d5573-48da-40ce-bd19-58ed1043941b service nova] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:32:20.991984 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1295/2598] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:32:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:32:22.853404 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=9 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:32:26.956996 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef886332-0233-4fa6-97be-2d6f3d4d69ba service nova] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:32:26.967473 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef886332-0233-4fa6-97be-2d6f3d4d69ba service nova] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:32:26.971040 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed91dad-a225-4533-b0d3-6b6680bedeee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:32:26.971676 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed91dad-a225-4533-b0d3-6b6680bedeee service neutron] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:32:26.972017 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed91dad-a225-4533-b0d3-6b6680bedeee service neutron] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:32:26.972291 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed91dad-a225-4533-b0d3-6b6680bedeee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:32:26.981674 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed91dad-a225-4533-b0d3-6b6680bedeee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:32:26.983196 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed91dad-a225-4533-b0d3-6b6680bedeee service neutron] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:32:26.992639 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1304/2599] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:32:26 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:32:27.332109 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287d5573-48da-40ce-bd19-58ed1043941b service nova] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:32:27.342367 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287d5573-48da-40ce-bd19-58ed1043941b service nova] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:32:27.346128 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0769aacc-e7ac-4e03-bfa7-22c657fe3de1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:32:27.346813 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0769aacc-e7ac-4e03-bfa7-22c657fe3de1 service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:32:27.347195 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0769aacc-e7ac-4e03-bfa7-22c657fe3de1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:32:27.347528 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0769aacc-e7ac-4e03-bfa7-22c657fe3de1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:32:27.353511 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-0769aacc-e7ac-4e03-bfa7-22c657fe3de1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:32:27.414415 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0769aacc-e7ac-4e03-bfa7-22c657fe3de1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:32:27.416448 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0769aacc-e7ac-4e03-bfa7-22c657fe3de1 service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:32:27.426322 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1296/2600] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:32:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:32:30.989263 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed91dad-a225-4533-b0d3-6b6680bedeee service neutron] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:32:30.990818 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-0ed91dad-a225-4533-b0d3-6b6680bedeee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:32:31.063033 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed91dad-a225-4533-b0d3-6b6680bedeee service neutron] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:32:31.067910 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcb8736-b5f5-44b0-a605-323cfe67e9e4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e528da752ed4c78bc747215909f6569', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:32:31.068512 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcb8736-b5f5-44b0-a605-323cfe67e9e4 service placement] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:32:31.068795 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcb8736-b5f5-44b0-a605-323cfe67e9e4 service placement] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:32:31.069067 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcb8736-b5f5-44b0-a605-323cfe67e9e4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:32:31.070783 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-cfcb8736-b5f5-44b0-a605-323cfe67e9e4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:32:31.082807 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcb8736-b5f5-44b0-a605-323cfe67e9e4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:32:31.084414 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcb8736-b5f5-44b0-a605-323cfe67e9e4 service placement] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:32:31.094440 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1305/2601] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:32:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:32:34.590194 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0769aacc-e7ac-4e03-bfa7-22c657fe3de1 service neutron] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:32:34.591724 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-0769aacc-e7ac-4e03-bfa7-22c657fe3de1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:32:34.606457 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0769aacc-e7ac-4e03-bfa7-22c657fe3de1 service neutron] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:32:34.609515 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c9d999-b485-4829-8351-de5f847f6723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:32:34.610141 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c9d999-b485-4829-8351-de5f847f6723 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:32:34.610414 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c9d999-b485-4829-8351-de5f847f6723 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:32:34.610681 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c9d999-b485-4829-8351-de5f847f6723 admin admin] PATH_INFO: `/v3/users/da2a3fcb0287474cb93238923aa9c7c1` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:32:34.611882 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c9d999-b485-4829-8351-de5f847f6723 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da2a3fcb0287474cb93238923aa9c7c1)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:32:34.613781 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c9d999-b485-4829-8351-de5f847f6723 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:32:34.831575 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-84c9d999-b485-4829-8351-de5f847f6723 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da2a3fcb0287474cb93238923aa9c7c1'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:32:34.832239 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-84c9d999-b485-4829-8351-de5f847f6723 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da2a3fcb0287474cb93238923aa9c7c1'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:32:34.844113 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-84c9d999-b485-4829-8351-de5f847f6723 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da2a3fcb0287474cb93238923aa9c7c1'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:32:34.919875 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-84c9d999-b485-4829-8351-de5f847f6723 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da2a3fcb0287474cb93238923aa9c7c1'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:32:34.920488 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-84c9d999-b485-4829-8351-de5f847f6723 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da2a3fcb0287474cb93238923aa9c7c1'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:32:34.934187 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1297/2602] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:32:34 2022] DELETE /identity/v3/users/da2a3fcb0287474cb93238923aa9c7c1 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:32:35.173388 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcb8736-b5f5-44b0-a605-323cfe67e9e4 service placement] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:32:35.185023 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcb8736-b5f5-44b0-a605-323cfe67e9e4 service placement] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:32:35.241684 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f0bee0-fad0-4bc8-95f5-41ed64703363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:32:35.242321 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f0bee0-fad0-4bc8-95f5-41ed64703363 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:32:35.242600 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f0bee0-fad0-4bc8-95f5-41ed64703363 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:32:35.242875 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f0bee0-fad0-4bc8-95f5-41ed64703363 admin admin] PATH_INFO: `/v3/projects/654843d065264b9e9296824daf702110` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:32:35.244622 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f0bee0-fad0-4bc8-95f5-41ed64703363 admin admin] RBAC: Authorizing `identity:delete_project(project_id=654843d065264b9e9296824daf702110)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:32:35.246575 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f0bee0-fad0-4bc8-95f5-41ed64703363 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:32:35.343289 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-59f0bee0-fad0-4bc8-95f5-41ed64703363 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '654843d065264b9e9296824daf702110', 'request_id': 'req-59f0bee0-fad0-4bc8-95f5-41ed64703363'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:32:35.349447 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-59f0bee0-fad0-4bc8-95f5-41ed64703363 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '654843d065264b9e9296824daf702110', 'request_id': 'req-59f0bee0-fad0-4bc8-95f5-41ed64703363'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:32:35.411349 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-59f0bee0-fad0-4bc8-95f5-41ed64703363 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '654843d065264b9e9296824daf702110', 'request_id': 'req-59f0bee0-fad0-4bc8-95f5-41ed64703363'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:32:35.422802 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-59f0bee0-fad0-4bc8-95f5-41ed64703363 admin admin] The token cache is being invalidate because project 654843d065264b9e9296824daf702110 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81747) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:32:35.423086 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-59f0bee0-fad0-4bc8-95f5-41ed64703363 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:32:35.423768 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-59f0bee0-fad0-4bc8-95f5-41ed64703363 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:32:35.425253 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1306/2603] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:32:35 2022] DELETE /identity/v3/projects/654843d065264b9e9296824daf702110 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:32:35.429307 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c9d999-b485-4829-8351-de5f847f6723 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:32:35.430707 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-84c9d999-b485-4829-8351-de5f847f6723 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:32:35.443684 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c9d999-b485-4829-8351-de5f847f6723 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:32:35.490113 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:32:35.498693 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305f002b-f7b9-4109-99ba-82afd92a84e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:32:35.499315 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305f002b-f7b9-4109-99ba-82afd92a84e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:32:35.499624 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305f002b-f7b9-4109-99ba-82afd92a84e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:32:35.499891 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305f002b-f7b9-4109-99ba-82afd92a84e9 admin admin] PATH_INFO: `/v3/users/269c39f34a014bcc87a1d47d2110105a` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:32:35.501004 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305f002b-f7b9-4109-99ba-82afd92a84e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=269c39f34a014bcc87a1d47d2110105a)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:32:35.502922 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305f002b-f7b9-4109-99ba-82afd92a84e9 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:32:35.687872 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-305f002b-f7b9-4109-99ba-82afd92a84e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '269c39f34a014bcc87a1d47d2110105a'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:32:35.688686 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-305f002b-f7b9-4109-99ba-82afd92a84e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '269c39f34a014bcc87a1d47d2110105a'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:32:35.701646 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-305f002b-f7b9-4109-99ba-82afd92a84e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '269c39f34a014bcc87a1d47d2110105a'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:32:35.766060 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-305f002b-f7b9-4109-99ba-82afd92a84e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '269c39f34a014bcc87a1d47d2110105a'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:32:35.766664 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-305f002b-f7b9-4109-99ba-82afd92a84e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '269c39f34a014bcc87a1d47d2110105a'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:32:35.781724 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1298/2604] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:32:35 2022] DELETE /identity/v3/users/269c39f34a014bcc87a1d47d2110105a => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:32:36.058968 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f0bee0-fad0-4bc8-95f5-41ed64703363 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:32:36.060396 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-59f0bee0-fad0-4bc8-95f5-41ed64703363 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:32:36.072861 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f0bee0-fad0-4bc8-95f5-41ed64703363 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:32:36.077036 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:32:36.133179 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2b1fef-5f86-4ebd-935f-845b8d57488a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:32:36.133814 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2b1fef-5f86-4ebd-935f-845b8d57488a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:32:36.134092 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2b1fef-5f86-4ebd-935f-845b8d57488a admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:32:36.134371 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2b1fef-5f86-4ebd-935f-845b8d57488a admin admin] PATH_INFO: `/v3/projects/406331653ce1431896c1d2ad89461310` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:32:36.135499 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2b1fef-5f86-4ebd-935f-845b8d57488a admin admin] RBAC: Authorizing `identity:delete_project(project_id=406331653ce1431896c1d2ad89461310)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:32:36.137974 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2b1fef-5f86-4ebd-935f-845b8d57488a admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:32:36.240538 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-2a2b1fef-5f86-4ebd-935f-845b8d57488a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '406331653ce1431896c1d2ad89461310', 'request_id': 'req-2a2b1fef-5f86-4ebd-935f-845b8d57488a'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:32:36.247211 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-2a2b1fef-5f86-4ebd-935f-845b8d57488a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '406331653ce1431896c1d2ad89461310', 'request_id': 'req-2a2b1fef-5f86-4ebd-935f-845b8d57488a'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:32:36.314120 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-2a2b1fef-5f86-4ebd-935f-845b8d57488a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '406331653ce1431896c1d2ad89461310', 'request_id': 'req-2a2b1fef-5f86-4ebd-935f-845b8d57488a'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:32:36.324957 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-2a2b1fef-5f86-4ebd-935f-845b8d57488a admin admin] The token cache is being invalidate because project 406331653ce1431896c1d2ad89461310 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81747) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:32:36.325243 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-2a2b1fef-5f86-4ebd-935f-845b8d57488a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:32:36.325872 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-2a2b1fef-5f86-4ebd-935f-845b8d57488a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:32:36.327344 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1307/2605] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:32:36 2022] DELETE /identity/v3/projects/406331653ce1431896c1d2ad89461310 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:32:45.776769 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=29,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:32:46.321429 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=48,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:34:14.146331 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305f002b-f7b9-4109-99ba-82afd92a84e9 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:34:14.148025 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-305f002b-f7b9-4109-99ba-82afd92a84e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:34:14.215299 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305f002b-f7b9-4109-99ba-82afd92a84e9 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:34:14.273369 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f8ba57-a37c-42d0-9653-52f1b0d3fcc8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e528da752ed4c78bc747215909f6569', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:34:14.273977 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f8ba57-a37c-42d0-9653-52f1b0d3fcc8 service placement] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:34:14.274254 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f8ba57-a37c-42d0-9653-52f1b0d3fcc8 service placement] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:34:14.274524 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f8ba57-a37c-42d0-9653-52f1b0d3fcc8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:34:14.276273 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-e4f8ba57-a37c-42d0-9653-52f1b0d3fcc8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:34:14.287333 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f8ba57-a37c-42d0-9653-52f1b0d3fcc8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:34:14.289009 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f8ba57-a37c-42d0-9653-52f1b0d3fcc8 service placement] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:34:14.299188 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1299/2606] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:34:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:34:24.294691 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=58 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:34:31.317100 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1188ed12-23ab-407a-8865-9a1111ccd6f9 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:34:31.317613 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1188ed12-23ab-407a-8865-9a1111ccd6f9 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:34:31.317729 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1188ed12-23ab-407a-8865-9a1111ccd6f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:34:31.336850 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-1188ed12-23ab-407a-8865-9a1111ccd6f9 None None] MFA Rules not processed for user `828b7770373f4e9688b0f218f7fd6e4f`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:34:31.338089 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-1188ed12-23ab-407a-8865-9a1111ccd6f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:34:31.392838 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1308/2607] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:34:31 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:34:31.698483 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f8ba57-a37c-42d0-9653-52f1b0d3fcc8 service placement] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:34:31.699919 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-e4f8ba57-a37c-42d0-9653-52f1b0d3fcc8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:34:31.754972 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f8ba57-a37c-42d0-9653-52f1b0d3fcc8 service placement] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:34:31.758597 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c93778e-94d0-4b01-9a62-5769d5258e8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:34:31.759192 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c93778e-94d0-4b01-9a62-5769d5258e8e service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:34:31.759498 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c93778e-94d0-4b01-9a62-5769d5258e8e service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:34:31.759771 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c93778e-94d0-4b01-9a62-5769d5258e8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:34:31.761480 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-6c93778e-94d0-4b01-9a62-5769d5258e8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:34:31.771232 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c93778e-94d0-4b01-9a62-5769d5258e8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:34:31.772948 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c93778e-94d0-4b01-9a62-5769d5258e8e service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:34:31.781443 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1300/2608] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:34:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:34:41.386202 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=25 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:34:41.777259 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=26 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:36:18.116602 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1188ed12-23ab-407a-8865-9a1111ccd6f9 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:36:18.127039 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1188ed12-23ab-407a-8865-9a1111ccd6f9 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:36:18.130552 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475a8c58-ca31-4ee1-b618-257bd7af08dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:36:18.131152 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a8c58-ca31-4ee1-b618-257bd7af08dc service neutron] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:36:18.131459 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a8c58-ca31-4ee1-b618-257bd7af08dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:36:18.131734 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a8c58-ca31-4ee1-b618-257bd7af08dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:36:18.133473 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-475a8c58-ca31-4ee1-b618-257bd7af08dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:36:18.142973 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475a8c58-ca31-4ee1-b618-257bd7af08dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:36:18.144427 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475a8c58-ca31-4ee1-b618-257bd7af08dc service neutron] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:36:18.152649 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1309/2609] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:36:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:36:28.149119 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=6 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:36:38.590906 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f7ea3-f233-463f-a967-e76492089a4d None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:36:38.591452 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f7ea3-f233-463f-a967-e76492089a4d None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:36:38.591582 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f7ea3-f233-463f-a967-e76492089a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:36:38.619776 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-403f7ea3-f233-463f-a967-e76492089a4d None None] MFA Rules not processed for user `828b7770373f4e9688b0f218f7fd6e4f`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 00:36:38.620970 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-403f7ea3-f233-463f-a967-e76492089a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:36:38.645004 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1301/2610] 10.222.0.36 () {58 vars in 958 bytes} [Wed Oct 12 00:36:38 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 00:36:38.685949 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475a8c58-ca31-4ee1-b618-257bd7af08dc service neutron] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:36:38.687531 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-475a8c58-ca31-4ee1-b618-257bd7af08dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:36:38.793667 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475a8c58-ca31-4ee1-b618-257bd7af08dc service neutron] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:36:38.797165 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d76160d-5d92-414b-91f3-42551def4021 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f87febbbaa94626807a98da4a771ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:36:38.797760 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d76160d-5d92-414b-91f3-42551def4021 service cinder] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:36:38.798032 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d76160d-5d92-414b-91f3-42551def4021 service cinder] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:36:38.798298 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d76160d-5d92-414b-91f3-42551def4021 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:36:38.799993 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-5d76160d-5d92-414b-91f3-42551def4021 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:36:38.875946 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d76160d-5d92-414b-91f3-42551def4021 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:36:38.877364 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d76160d-5d92-414b-91f3-42551def4021 service cinder] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:36:38.888137 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1310/2611] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:36:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:36:39.868614 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403f7ea3-f233-463f-a967-e76492089a4d None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:36:39.870122 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-403f7ea3-f233-463f-a967-e76492089a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:36:39.883024 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d76160d-5d92-414b-91f3-42551def4021 service cinder] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:36:39.884537 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-5d76160d-5d92-414b-91f3-42551def4021 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:36:39.886773 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403f7ea3-f233-463f-a967-e76492089a4d None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:36:39.892722 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614cc652-7390-499b-a990-f37f323b4c07 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e528da752ed4c78bc747215909f6569', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:36:39.893368 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614cc652-7390-499b-a990-f37f323b4c07 service placement] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:36:39.893646 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614cc652-7390-499b-a990-f37f323b4c07 service placement] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:36:39.893915 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614cc652-7390-499b-a990-f37f323b4c07 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:36:39.895763 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-614cc652-7390-499b-a990-f37f323b4c07 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:36:39.909521 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614cc652-7390-499b-a990-f37f323b4c07 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:36:39.911147 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614cc652-7390-499b-a990-f37f323b4c07 service placement] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:36:39.921373 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d76160d-5d92-414b-91f3-42551def4021 service cinder] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:36:39.921624 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1302/2612] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:36:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:36:39.925090 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b363e878-99a5-494b-b9e6-dd7cd879c38c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:36:39.925693 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b363e878-99a5-494b-b9e6-dd7cd879c38c service neutron] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:36:39.925968 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b363e878-99a5-494b-b9e6-dd7cd879c38c service neutron] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:36:39.926235 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b363e878-99a5-494b-b9e6-dd7cd879c38c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:36:39.928099 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-b363e878-99a5-494b-b9e6-dd7cd879c38c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:36:39.942272 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b363e878-99a5-494b-b9e6-dd7cd879c38c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:36:39.943764 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b363e878-99a5-494b-b9e6-dd7cd879c38c service neutron] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:36:39.956941 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1311/2613] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:36:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 00:36:46.914996 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614cc652-7390-499b-a990-f37f323b4c07 service placement] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:36:46.916510 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-614cc652-7390-499b-a990-f37f323b4c07 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:36:46.932280 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614cc652-7390-499b-a990-f37f323b4c07 service placement] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:36:46.937620 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab57f7ff-1103-4fa4-8a92-5417fdb3038a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a515b75aa3944912ad2364ffd7924f40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:36:46.938208 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab57f7ff-1103-4fa4-8a92-5417fdb3038a service nova] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:36:46.938486 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab57f7ff-1103-4fa4-8a92-5417fdb3038a service nova] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:36:46.938757 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab57f7ff-1103-4fa4-8a92-5417fdb3038a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:36:46.940417 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-ab57f7ff-1103-4fa4-8a92-5417fdb3038a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:36:46.950234 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab57f7ff-1103-4fa4-8a92-5417fdb3038a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:36:46.951822 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab57f7ff-1103-4fa4-8a92-5417fdb3038a service nova] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:36:46.960465 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1303/2614] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:36:46 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:36:49.950017 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=94 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:36:55.507145 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b363e878-99a5-494b-b9e6-dd7cd879c38c service neutron] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:36:55.519155 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b363e878-99a5-494b-b9e6-dd7cd879c38c service neutron] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:36:55.524879 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184a0c70-0261-486e-9d70-6e9e5d8e2921 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a515b75aa3944912ad2364ffd7924f40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:36:55.526159 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184a0c70-0261-486e-9d70-6e9e5d8e2921 service nova] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:36:55.526159 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184a0c70-0261-486e-9d70-6e9e5d8e2921 service nova] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:36:55.526159 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184a0c70-0261-486e-9d70-6e9e5d8e2921 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:36:55.527754 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-184a0c70-0261-486e-9d70-6e9e5d8e2921 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:36:55.540418 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184a0c70-0261-486e-9d70-6e9e5d8e2921 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:36:55.542129 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184a0c70-0261-486e-9d70-6e9e5d8e2921 service nova] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:36:55.553690 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1312/2615] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:36:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:36:56.956601 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=27 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:37:05.548998 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=6 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:38:40.593678 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab57f7ff-1103-4fa4-8a92-5417fdb3038a service nova] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:38:40.601489 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-ab57f7ff-1103-4fa4-8a92-5417fdb3038a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:38:40.622543 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab57f7ff-1103-4fa4-8a92-5417fdb3038a service nova] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:38:40.629468 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbc3285-18fc-4bc2-b12d-538a146916f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a515b75aa3944912ad2364ffd7924f40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:38:40.630188 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbc3285-18fc-4bc2-b12d-538a146916f4 service nova] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:38:40.630544 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbc3285-18fc-4bc2-b12d-538a146916f4 service nova] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:38:40.630888 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbc3285-18fc-4bc2-b12d-538a146916f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:38:40.634142 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-3cbc3285-18fc-4bc2-b12d-538a146916f4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:38:40.647761 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbc3285-18fc-4bc2-b12d-538a146916f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:38:40.649865 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbc3285-18fc-4bc2-b12d-538a146916f4 service nova] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:38:40.663100 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1304/2616] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:38:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:38:43.534493 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184a0c70-0261-486e-9d70-6e9e5d8e2921 service nova] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:38:43.552745 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184a0c70-0261-486e-9d70-6e9e5d8e2921 service nova] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:38:43.558233 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c66a421-dc60-4ee3-8bce-1e724326252d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a515b75aa3944912ad2364ffd7924f40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:38:43.558832 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66a421-dc60-4ee3-8bce-1e724326252d service nova] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:38:43.559105 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66a421-dc60-4ee3-8bce-1e724326252d service nova] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:38:43.559395 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66a421-dc60-4ee3-8bce-1e724326252d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:38:43.561112 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-7c66a421-dc60-4ee3-8bce-1e724326252d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:38:43.573481 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c66a421-dc60-4ee3-8bce-1e724326252d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:38:43.575074 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c66a421-dc60-4ee3-8bce-1e724326252d service nova] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:38:43.586223 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1313/2617] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:38:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:38:50.657841 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=9 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:38:53.581752 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=9 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:38:56.090609 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbc3285-18fc-4bc2-b12d-538a146916f4 service nova] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:38:56.100821 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbc3285-18fc-4bc2-b12d-538a146916f4 service nova] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:38:56.105239 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915513ec-6976-4c30-8016-d51adeafca46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51215dab618141b4991e232724094140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:38:56.105239 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915513ec-6976-4c30-8016-d51adeafca46 service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:38:56.105468 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915513ec-6976-4c30-8016-d51adeafca46 service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:38:56.105642 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915513ec-6976-4c30-8016-d51adeafca46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:38:56.107417 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-915513ec-6976-4c30-8016-d51adeafca46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:38:56.194330 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915513ec-6976-4c30-8016-d51adeafca46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:38:56.195805 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915513ec-6976-4c30-8016-d51adeafca46 service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:38:56.214773 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1305/2618] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:38:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:38:59.640900 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c66a421-dc60-4ee3-8bce-1e724326252d service nova] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:38:59.663482 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c66a421-dc60-4ee3-8bce-1e724326252d service nova] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:38:59.668605 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6342980-06bc-4278-867d-007dc00aaafc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e528da752ed4c78bc747215909f6569', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:38:59.670575 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6342980-06bc-4278-867d-007dc00aaafc service placement] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:38:59.670575 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6342980-06bc-4278-867d-007dc00aaafc service placement] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:38:59.670575 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6342980-06bc-4278-867d-007dc00aaafc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:38:59.672043 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-c6342980-06bc-4278-867d-007dc00aaafc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:38:59.696690 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6342980-06bc-4278-867d-007dc00aaafc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:38:59.698263 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6342980-06bc-4278-867d-007dc00aaafc service placement] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:38:59.708432 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1314/2619] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:38:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:39:03.321160 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915513ec-6976-4c30-8016-d51adeafca46 service neutron] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:39:03.322639 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-915513ec-6976-4c30-8016-d51adeafca46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:39:03.344963 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915513ec-6976-4c30-8016-d51adeafca46 service neutron] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:39:03.344963 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf0a953-5762-4530-82cd-626d8bea6ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:39:03.344963 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf0a953-5762-4530-82cd-626d8bea6ec3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:39:03.344963 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf0a953-5762-4530-82cd-626d8bea6ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:39:03.345643 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf0a953-5762-4530-82cd-626d8bea6ec3 admin admin] PATH_INFO: `/v3/users/828b7770373f4e9688b0f218f7fd6e4f` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:39:03.345643 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf0a953-5762-4530-82cd-626d8bea6ec3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=828b7770373f4e9688b0f218f7fd6e4f)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:39:03.346985 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf0a953-5762-4530-82cd-626d8bea6ec3 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:39:03.542388 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-abf0a953-5762-4530-82cd-626d8bea6ec3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '828b7770373f4e9688b0f218f7fd6e4f'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:39:03.543063 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-abf0a953-5762-4530-82cd-626d8bea6ec3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '828b7770373f4e9688b0f218f7fd6e4f'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:39:03.555514 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-abf0a953-5762-4530-82cd-626d8bea6ec3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '828b7770373f4e9688b0f218f7fd6e4f'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:39:03.625027 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-abf0a953-5762-4530-82cd-626d8bea6ec3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '828b7770373f4e9688b0f218f7fd6e4f'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:39:03.625910 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-abf0a953-5762-4530-82cd-626d8bea6ec3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '828b7770373f4e9688b0f218f7fd6e4f'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:39:03.639145 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1306/2620] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:39:03 2022] DELETE /identity/v3/users/828b7770373f4e9688b0f218f7fd6e4f => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:39:03.869328 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6342980-06bc-4278-867d-007dc00aaafc service placement] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:39:03.881040 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6342980-06bc-4278-867d-007dc00aaafc service placement] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:39:03.938843 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17c8a44-7425-43c1-a021-ecb4bfbd2c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:39:03.939514 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17c8a44-7425-43c1-a021-ecb4bfbd2c2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:39:03.939800 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17c8a44-7425-43c1-a021-ecb4bfbd2c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:39:03.940070 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17c8a44-7425-43c1-a021-ecb4bfbd2c2b admin admin] PATH_INFO: `/v3/projects/764ed0f4ff5d4c48b89dc4be9460323b` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:39:03.941204 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17c8a44-7425-43c1-a021-ecb4bfbd2c2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=764ed0f4ff5d4c48b89dc4be9460323b)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:39:03.943095 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17c8a44-7425-43c1-a021-ecb4bfbd2c2b admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:39:04.034961 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-e17c8a44-7425-43c1-a021-ecb4bfbd2c2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '764ed0f4ff5d4c48b89dc4be9460323b', 'request_id': 'req-e17c8a44-7425-43c1-a021-ecb4bfbd2c2b'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:39:04.041568 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-e17c8a44-7425-43c1-a021-ecb4bfbd2c2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '764ed0f4ff5d4c48b89dc4be9460323b', 'request_id': 'req-e17c8a44-7425-43c1-a021-ecb4bfbd2c2b'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:39:04.116683 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-e17c8a44-7425-43c1-a021-ecb4bfbd2c2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '764ed0f4ff5d4c48b89dc4be9460323b', 'request_id': 'req-e17c8a44-7425-43c1-a021-ecb4bfbd2c2b'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:39:04.128206 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-e17c8a44-7425-43c1-a021-ecb4bfbd2c2b admin admin] The token cache is being invalidate because project 764ed0f4ff5d4c48b89dc4be9460323b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81747) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:39:04.128491 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-e17c8a44-7425-43c1-a021-ecb4bfbd2c2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:39:04.129192 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-e17c8a44-7425-43c1-a021-ecb4bfbd2c2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:39:04.130661 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1315/2621] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:39:03 2022] DELETE /identity/v3/projects/764ed0f4ff5d4c48b89dc4be9460323b => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:39:04.134807 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf0a953-5762-4530-82cd-626d8bea6ec3 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:39:04.136245 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-abf0a953-5762-4530-82cd-626d8bea6ec3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:39:04.149433 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf0a953-5762-4530-82cd-626d8bea6ec3 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:39:04.182513 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:39:04.204333 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea8c196-8902-48bd-bcc8-1198b3e612ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:39:04.204959 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea8c196-8902-48bd-bcc8-1198b3e612ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:39:04.205236 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea8c196-8902-48bd-bcc8-1198b3e612ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:39:04.205510 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea8c196-8902-48bd-bcc8-1198b3e612ba admin admin] PATH_INFO: `/v3/users/306f020dfd444edba42f023d742cd1c3` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:39:04.206753 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea8c196-8902-48bd-bcc8-1198b3e612ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=306f020dfd444edba42f023d742cd1c3)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:39:04.208695 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea8c196-8902-48bd-bcc8-1198b3e612ba admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:39:04.413384 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-9ea8c196-8902-48bd-bcc8-1198b3e612ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '306f020dfd444edba42f023d742cd1c3'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:39:04.414037 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-9ea8c196-8902-48bd-bcc8-1198b3e612ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '306f020dfd444edba42f023d742cd1c3'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:39:04.426160 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-9ea8c196-8902-48bd-bcc8-1198b3e612ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '306f020dfd444edba42f023d742cd1c3'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:39:04.484725 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-9ea8c196-8902-48bd-bcc8-1198b3e612ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '306f020dfd444edba42f023d742cd1c3'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:39:04.485341 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-9ea8c196-8902-48bd-bcc8-1198b3e612ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '306f020dfd444edba42f023d742cd1c3'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:39:04.497615 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1307/2622] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Oct 12 00:39:04 2022] DELETE /identity/v3/users/306f020dfd444edba42f023d742cd1c3 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:39:04.764452 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17c8a44-7425-43c1-a021-ecb4bfbd2c2b admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:39:04.765892 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-e17c8a44-7425-43c1-a021-ecb4bfbd2c2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:39:04.780835 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17c8a44-7425-43c1-a021-ecb4bfbd2c2b admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:39:04.832934 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:39:04.838945 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4dc078-12ed-4455-9a30-000895cad212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fcb9022bfe743ae9405fc8470a5e016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392dba9b727b479aa54a2746402ded73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:39:04.839645 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4dc078-12ed-4455-9a30-000895cad212 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:39:04.839922 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4dc078-12ed-4455-9a30-000895cad212 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:39:04.840193 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4dc078-12ed-4455-9a30-000895cad212 admin admin] PATH_INFO: `/v3/projects/9082ff8249a9467abcfb4708046a18c6` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:39:04.841356 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4dc078-12ed-4455-9a30-000895cad212 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9082ff8249a9467abcfb4708046a18c6)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:39:04.843289 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4dc078-12ed-4455-9a30-000895cad212 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:39:04.945279 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-cd4dc078-12ed-4455-9a30-000895cad212 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9082ff8249a9467abcfb4708046a18c6', 'request_id': 'req-cd4dc078-12ed-4455-9a30-000895cad212'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:39:04.952125 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-cd4dc078-12ed-4455-9a30-000895cad212 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9082ff8249a9467abcfb4708046a18c6', 'request_id': 'req-cd4dc078-12ed-4455-9a30-000895cad212'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:39:05.014352 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-cd4dc078-12ed-4455-9a30-000895cad212 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9082ff8249a9467abcfb4708046a18c6', 'request_id': 'req-cd4dc078-12ed-4455-9a30-000895cad212'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:39:05.024932 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-cd4dc078-12ed-4455-9a30-000895cad212 admin admin] The token cache is being invalidate because project 9082ff8249a9467abcfb4708046a18c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81747) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 00:39:05.025216 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-cd4dc078-12ed-4455-9a30-000895cad212 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:39:05.025852 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-cd4dc078-12ed-4455-9a30-000895cad212 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 00:39:05.027305 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1316/2623] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Oct 12 00:39:04 2022] DELETE /identity/v3/projects/9082ff8249a9467abcfb4708046a18c6 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 00:39:14.651268 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=36,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:39:15.099689 dv-cinder-849143-13-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=25,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 00:39:20.565140 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea8c196-8902-48bd-bcc8-1198b3e612ba admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 00:39:20.566630 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-9ea8c196-8902-48bd-bcc8-1198b3e612ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:39:20.634356 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea8c196-8902-48bd-bcc8-1198b3e612ba admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 00:39:20.692507 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce19801f-ede9-45c7-b2f1-bb83fbe0990c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e528da752ed4c78bc747215909f6569', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0697927c6aac43d4b14a750fbbb1145e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 00:39:20.693107 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce19801f-ede9-45c7-b2f1-bb83fbe0990c service placement] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 00:39:20.693382 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce19801f-ede9-45c7-b2f1-bb83fbe0990c service placement] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 00:39:20.693649 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce19801f-ede9-45c7-b2f1-bb83fbe0990c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 00:39:20.695358 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-ce19801f-ede9-45c7-b2f1-bb83fbe0990c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 00:39:20.706259 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce19801f-ede9-45c7-b2f1-bb83fbe0990c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 00:39:20.707860 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce19801f-ede9-45c7-b2f1-bb83fbe0990c service placement] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 00:39:20.717593 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1308/2624] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Oct 12 00:39:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 00:39:30.713322 dv-cinder-849143-13-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=58 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}